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(57) Abstract: A multi-planar volumetric display system (10) and method generate volumetric three-dimensionaJ images (34 and 56) 
using a multi-surface optical device (32) including a plurality of individual optical elements (36-40) corresponding to a plurality of 
image depths. An image projector (20) selectively projects images (24-30, 44-50) on respective optical elements (36-40) to generate 
a volumetric three-dimensional image (34) viewable in the multi-surface optical device (32). Psychological vision cues are added 
during generation of the three-dimensional image (34) to enhance depth perception when viewing the three-dimensional image. 
A floating-image generator (54) may project the three-dimensional image having enhanced depth to generate a second volumetric 
three-dimensional image (56) viewable as floating in space. 
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MULTI-PLANAR VOLUMETRIC DISPLAY SYSTEM AND METHOD 
OF OPERATION USING THREE-DIMENSIONAL ANTI-ALIASING 

CROSS-REFERENCE TO RELATED APPUCATION 
5 [ 0001] This application claims the benefit of U.S. Application No. 
09/933,424, filed August 20, 2001. 

BACKGROUND OF THE INVENTION 
[ 0002] The present invention relates to three-dimensional (3D) imaging, 
and, more particularly, to a multi-planar display system using 3D anti-aliasing 

10 for generating volumetric three-dimensional images in space. 

[ 0003] It is known that three-dimensional (3D) images may be generated 
and viewed to appear in space. Typically, specialized eyewear such as goggle 
and/or helmets are used, but such eyewear can be encumbering. In addition, 
by its nature as an accessory to the eyes, such eyewear reduces the perception of 

15 viewing an actual 3D image. Also, llie use of such eyewear can cause eye 
fatigue which is remedied by limiting the time to view tiie image, and such 
eyewear is often bulky and uncomfortable to wear. 
[ 0004] Thus, there is a need to generate volumetric 3D images and 
displays without die disadvantages of using such eyewear. 

20 [ 0005] Other volumetric systems generate such volumetric 3D images 
using, for example, self-luminescent volume elements, that is, voxels. Before 
providing examples of such systems, it is important to distinguish the much 
abused term 'Voxel" from a 3D data element (referred to herein as a "tridel"). A 
voxel is the actual glowing point of light in a 3D display and is analogous to a 

25 pixel in a 2D display. However, a tiidel is an abstract 3D data type. More 
specifically, voxels have positions that are integers (i, j, k) and only have the 
properties of color and brighmess, whereas Oidels are characterized by a set of 
parameters defined at a floating point location (x, y, z) in a virtiial image space. 
Thus, in its most general sense, a tridel is a 3D data type any may encompass 
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any number of appUcation-specific data types. For example, if the tridel is used 
to define polygonal vertices of a 3D object then the data parameters of this 
abstract 3D data type are color (R, G, B) and visual opacity (A). As another 
example, if the tiidel represents a data element of an image produced by a 
5 medical computed x-ray tomography ("CT) scanner, then the data parameter is 
x-ray opacity. In yet another example, if the tridel describes a thermonuclear 
plasma, then the data parameters might be plasma density, temperature, and 
average velocity of the plasma constituents. 

[ 0006] From the foregoing, it will be understood that to produce an 

10 image, either 2D or 3D, each tridel must be mathematicaUy processed into a 
pixel or voxel. This processing may include geometric transformations including 
rotation, scaling, stretching or compression, perspective, projection and 
viewpoint transformations, all of which operate on the x, y, z coordinates of the 
tridel. Further, in the process of determining ±e color and brightness of a pixel 

15 or voxel, tridels may be averaged together when there are many within the 
space of one voxel or interpolated between when there many pixels widiin the 
space of two tridels. The distinction between tiidels and voxels will be more 
clearly appreciated upon consideration of the depth transformation discussed 
below for mapping the depth coordinate of a tridel into the voxel depth 

20 coordinate within the MOE device 32. 

[ 0007] Turning to examples of other volumetric display systems known in 
the art, one example of a volumetric image system is the system of 3D 
TECHNOLOGY LABORATORIES of Mountain View, Calif., in which the 
mtersection of infrared laser beams in a solid glass or plastic volume doped with 

25 rare earth impurity ions generates such voxel-based images. However, the non- 
linear effect that creates visible Ught from two invisible infrared laser beams has 
a very low efficiency of about 1%, which results in die need for powerful lasers 
to create a bright image in a large display. Such powerful lasers are a potential 
eye hazard requiring a significant protective enclosure around the display. 

30 Additionally, scanned lasers typically have poor resolution resulting in low voxel 
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count, and the soUd nature of the voliunetric mechanism results in large massive 
systems that are very heavy. 

[ 0008] Another volumetric display system from ActuaUty Systems, Inc. of 
, Cambridge, Massachusetts, uses a linear array of laser diodes that are reflected 
5 off of a rapidly spinning muWfaceted mirror onto a rapidly spuming projection 
screen. However, such rapidly spinning components, which may be relatively 
large in size, must be carefully balanced to avoid vibration and possibly 
catastrophic failure. AdditionaUy, the size, shape, and orientation of voxels 
within the display depends on their location, resulting in a position-dependent 

10 display resolution. 

[ 0009] Another volumetric display system is provided by NEOS 
TECHNOLOGIES, INC., of Melbourne, Florida, which scans a laser beam 
acousto-optically onto a rapidly spinning helical projection screen. Such a large 
spinning component requures a carefully maintained balance independent of 

15 display motion. The laser scanner system has poor resolution and low speed, 
drastically limiting the number of voxels. Additionally, die size, shape, and 
orientation of voxels witWn the display depends on tiieir location, resulting in a 
position-dependent resolution. Finally, the dramaticaUy non-rectiUnear nature 
of die display greatiy increases die processing requirements to calculate die 

20 different two-dimensional images. 

[ 0010] Other types of 3D imaging system are known, such as stereoscopic 
displays, which provide each eye widi a sUghtiy different perspective view of a 
scene. The brain tiien fuses the separate images into a single 3D image. Some 
systems provide only a single viewpoint and require special eyewear, or may 

25 perfom headtracking to eliminate eyewear but tiien the 3D image can be seen 
by only a single viewer. Alternatively, the display may provide a multitude of 
viewing zones at different angles with the image in each zone appropriate to 
that point of view, such as multi-view autostereoscopic displays. The eyes of the 
user must be within separate but adjacent viewing zones to see a 3D image, and 

30 die viewing zone must be very narrow to prevent a disconcerting jumpiness as 
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the Viewer moves relative to the display. Some systems have only horizontal 
parallax/lookaround. In addition, depth fbcusing-convergence disparity can 
rapidly lead to eyestrain that strongly limits viewing time. Additionally, 
stereoscopic displays have a limited field of view and cannot be used 
5 realistically with direct interaction technologies such as virtual reality and/or a 
force feedback interface. 

[ 00111 Headmounted displays (HMD) are typically employed in virtual 
reality applications, in which a pair of video displays present appropriate 
perspective views to each eye. A single HMD can only be used by one person at 
10 a time, and provide each eye with a limited field of view. Headtracldng must be 
used to provide parallax. 

[ 0012] Other display systems include holographic displays, in which the 
image is created through the interaction of coherent laser light with a pattern of 
very fine lines known as a holographic grating. The gracing alters the direction 

15 and intensity of the incident light so that it appears to come from the location of 
die objects being displayed. However, a typical optical hologram contains an 
enormous amount of information, so. updating a holographic display at high 
rates is computationally intensive. For a holographic display having a relatively 
large size and sufficient field of view, the pixel count is generally greater than 

20 250 million. 

[ 0013] Accordingly, a need exists for high quaUty volumetric 3D unaging 
widi computationally acceptable demands on processing systems and which has 
improved viewability and implementation. 

[ 0014] In addition, in tiiree-dimensional hnaging, the use of discrete 
25 voxels renders portions of images to appear jagged due to pbcelization, for 

example, for features at transitions between discrete deptiis in a volumetric 3D 
image. A need exists for a metfiod which softens die transition between 
portions of a volumetric 3D image. 
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SUMMARY OF THE INVENTION 
[ 0015] A multi-planar volumetric display (MVD) system and method of 
operation are disclosed which generate volumetric three-dimensional images. 
The MVD system includes a multi-surface optical device including a plurality of 
5 individual optical elements arranged in an array; an image projector for 

selectively projecting a set of images on respective optical elements of the multi- 
surface optical device; and a floating-image generator for projecting the first 
volumetric three-dimensional image from the multi-surface optical devices to 
generate a second volumetric three-dimensional image viewable as floating in 
10 space at a location separate from the multi-surface optical device. 

[ 00 1 6] Each of the plurality of the individual optical elements of the 
multi-surface optical device includes a liquid crystal element having a 
controllable variable translucency. An optical element controUer is also 
provided for controUing the translucency of tiie liquid crystal elements, such 
15 tiiat a single Uquid crystal element is controlled to have an opaque light- 
scattering state to receive and display the respective one of die set of images 
from die image projector, and tiie remaining Uquid crystal elements are 
controlled to be substantially transparent to aUow tiie viewing of die displayed 
unage on the opaque liquid crystal element. 
20 [ 0017] The optical element conti^Uer rasters tiuough the Uquid crystal 
elements at a high rate during a plurality of unaging cycles to select one Uquid 
crystal element tiierefrom to be in tiie opaque light-scattering state during a 
particular imaging cycle, and to cause the opaque Ught-scattering state to move 
tiu-ough die Uquid crystal elements for successively receiving die set of images 
25 and for generating die volumetric three-dimensional images with tiiree- 
dimensional depth. 

[ 0018] The image projector projects die set of images into die multi- 
surface optical device to generate the entire first volumetric duree-dimensional 
image in die multi-surface optical device at a rate greater tiian 35 Hz to prevent 
30 human-perceivable image flicker. For example, die volume rate may be about 
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40 Hz. In one embodiment, for example, if about 50 optical elements are used 
with a volume rate of about 40 Hz, the image projector projects each of the set 
of images onto a respective optical element at a rate of 2 kHz. 
[ 0019] The image projector includes a projection lens for outputting the 

5 set of images. The projector also includes an adaptive optical focusing system 
for focusing each of the set of images on die respective optical elements to 
control the resolution and depth of the projection of the set of unages from die 
projection lens. Alternatively or in addition, the image projector includes a 
plurality of laser light sources for projecting red, green, and blue laser light, 

10 respectively, to generate and project die set of images in a plurality of colors. 
[ 0020] In addition, a 3D anti-aliasing method is employed to smootii the 
portions of the projected images at transitions between optical elements in the 
multi-surface optical device. The anti-aliasing adjusts ±e display of voxels in a 
transition between optical elements, such that color values of the voxels are 

15 modified as a function of die distance of the voxels from the optical elements, to 
generate a smootii transition between the portions of the volumetric three- 
dimensional image. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[ 0021] FIG. 1 illustrates the disclosed multi-planar volumetric display 
20 system; 

[ 0022] FIG. 2 illustrates a liquid crystal based optical element having a 
transparent state; 

[ 0023] FIG. 3 illustrates die optical element of FIG. 2 in a scattering 
opaque state; 

25 [ 0024] FIGS. 4-7 illustrate successive displays of images on multiple 
optical elements to form a volumetric 3D image; 
[ 0025] FIG. 8 illustrates a membrane light modulator; 
[ 0026] FIG. 9 illustrates an adaptive optics system used in an image ^ 
projector; 
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[ 0027] FIG. 10 illustrates the adaptive optics system of FIG. 9 in 
conjunction with a multiple optical element system; 
[ 0028] FIG. 11 illustrates a side cross-sectional view of a pixel of a 
ferroelectric liquid crystal (FLQ spatial Ught modulator (SIM); 
5 [ 0029] FIGS. 12-14 illustrate angular orientations of the axes of the FLC 
SLMofFIG. 11; 

[ 0030] FIG. 15 illustrates a flow chart of a method for generating a multi- 
planar dataset; 

[ 0031] FIG. 16 illustrates 3D anti-aliasing of a voxel in a plurality of 

10 optical elements; 

[ 0032] FIG. 17 illustrates voxel display without 3D anti-aliasing 
[ 0033] FIG. 18 illustrates voxel display with 3D anti-aliasing 
[ 0034] FIG. 19 illustrates a graph comparing apparent depth witii and 
witiiout 3D anti-aliasing; 

15 [ 0035] FIG. 20 illustrates a flow chart of a method implementing 3D anti- 
aliasing; 

[ 0036] FIGS. 21-22 illustrate tiie generation of 3D images having 
translucent foreground objects witiiout anti-aliasing; and 
[ 0037] FIGS. 23-24 illustrate tiie generation of 3D images having 
20 translucent foreground objects with anti-aliasing. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[ 0038] Referring now to FIG. 1, a multi-planar volumetric display system 
10 is provided which generates three-dimensional (3D) images which are 
volumetric in nature, diat is, the 3D images occupy a definite and lunited 
25 volume of 3D space, and so exist at die location where die images appear. 
Thus, such 3D images are true 3D, as opposed to an image perceived to be 3D 
due to an optical illusion of vision such as by stereographic metiiods. 
[ 00391 The 3D images generated by die system 10 can have a very high 
resolution and can be displayed in a large range of colors, and so can have die 
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characteristics associated with viewing a real object. For example, such 3D 
images may have both horizontal and vertical motion paraUax or lookaround, 
allowing the viewer 12 to move yet still receive visual cues to maintain the 3D 
appearance of the 3D images. 

5 [ 0040] In addition, a viewer 12 does not need to wear any special 
eyewear such as stereographic visors or glasses to view the 3D image, which is 
advantageous since such eyewear is encumbering, causes eye fatigue, etc. 
Furtiiermore, the 3D image has a continuous field of view both horizontally and 
vertically, with the horizontal field of view equal to 360* in certain display 
10 configurations. Additionally, the viewer can be at any arbiti-ary viewing 
distance fi-om ±e MVD system 10 vdthout loss of 3D perception. 
[ 0041] The multi planar volumetric display system 10 includes an 
interface 14 for receiving 3D graphics data from a graphics data source 16, such 
as a computer which may be incorporated into die system 10, or which may be 

15 operatively connected to the system 10 through communications channels from, 
for example, a remote location and ronnected over conventional 
telecommunications links or over any network such as the Internet. The 
interface 14 may be a PCI bus, or an accelerated graphics port CAGP) interface 
avaUable from INTEL of Santa Clara, California. Otiier interfece may be used, 

20 such as the VME backplane interconnection bus system standardized as die IEEE 
1014 standard, the Small Computer System Interfiace (SCSD, the NuBus high- 
performance expansion bus system used ua Apple Macintosh computers and 
other systems, as weU as the Industry Standard Architecture QSA) interface, the 
Extended ISA (EISA) interface, die Universal Serial Bus (USB) interface, the 

25 FireWire bus interface now standardized as die IEEE 1394 standard offering 
high-speed communications and isochronous real-time data services in 
computers, as well as open or proprietary interfaces. 
[ 0042] The interface 14 passes tiie 3D graphics data to a multi-planar 
volumetric display (MVD) controller 18, which includes a large high speed 

30 image buffer. The three-dimensional image to be viewed as a volumetric 3D 
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image is converted by the MVD controller 18 into a series of two-dimensional 
image slices at varying depths through the 3D image. The frame data 
corresponding to the image slices are then rapidly output from the high speed 
image buffer of the MVD confroller 18 to an unage projector 20. 

5 [ 0043] The MVD controller 18 and the mterface 14 may be implemented 
in a computer, such as an OCTANE graphics workstation commerciaUy avaUable 
from SIUCON GRAPHICS of Mountain View, California. Other general 
computer-based systems may also be used, such as a personal computer (PQ 
using, for example, a 195 MHZ reduced instruction set computing (RISC) 

10 microprocessor. Accordingly, it is to be understood that the disclosed MVD 
system 10 and its components are not limited to a particular implementation or 
realization of hardware and/or software. 

[ 0044] The graphics data source 16 may optionally be a graphics 
apphcation program of a computer which operated an application program 

15 interface (API) and a device driver for providing the 3D image data in an 
appropriate format to die MVD conttoUer 18 of tiie computer tfirough an 
input/output (I/O) device such as die interface 14. The MVD controUer 18 may 
be hardware and/or software, for example, implemented m a personal 
computer and optionally using expansion cards for speciaUzed data processing. 

20 [ 0045] For example, an expansion card hi the MVD controller 18 may 
include graphics hardware and/or software for converting the 3D dataset from 
the graphics data source 16 into ±e series of two-dimensional unage slices 
forming a multi-planar dataset corresponding to the slices 24-30. Thus die 3D 
image 34 is generate at a real-time or near-real-time update rates for real world 

25 applications such as surgical stimulation, air traffic confrol, or military 
. command and control. Such expansion cars may also include a geometry 
engine for manipulating 3D datasets and texture memory for doing die texture 
mappuig of the 3D images. 

[ 0046] Prior to transmission of the image data to the image projector 20, 
30 die MVD contiroller 18 or alternatively die graphics data source 16 may perform 
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3D anti-aliasing on the image data to smooth the features to be displayed in the 
3D image 34, and so to avoid any jagged lines in depth, for example, between 
parallel planes along the z-direction, due to display pixelization caused by the 
inherently discrete voxel construction of tiie MOE device 32 with the optical 

5 elements 36-42 aUgned in the x-y planes normal to a z-axis. As tiie data 
corresponding to the image slices 24-30 is generated, an image element may 
appear near an edge of a plane transition, that is, between optical elements, for 
example, the optical elements 36-38. to avoid an abrupt transition at the 
specific image element, both of sUces 24, 26 may be generated such that each of 

10 the images 44-46 includes the specific image element, and so the image element 
is shared between both planes formed by die optical elements 36-38, which 
softens the transition and allows tiie 3D image 34 to appear more continuous. 
The brighmess of the image elements on respective consecutive optical elements 
is varied in accordance with the location of the image element in the image 

15 data. 

[ 0047] The graphics data source 16 and die MVD controller 18 may also 
perform zero-run encoding dirough die interface 14 in order to maximize the 
rate of tiransfer of image data to the MVD controller 18 for image generation. It 
is to be understood that otiier techniques for tiransferring the image data may be 

20 employed, such as the Motion Hcture Experts Group (MPEG) data 
communication standards as well as delta (r) compression. 
[ 0048] A 3D image may contain on die order of 50 SVGA resolution 
images updated at a rate of 40 Hz, which results in a raw data rate of more tiian 
2 GB/sec. To be displayed. Such a raw data rate can be significantiy reduced by 

25 transmitting zeros. A volumetric 3D image is typically represented by a large 
number of zeros associated witii the inside of objects, background objects, 
obstructed by foreground objects, and surrounding empty space. The graphics 
source 16 may encode die image data such tiiat a run of zeros is represented by 
a zero-run flag (ZRF) or zero-run code, and followed by or associated with a 

30 run lengdi. Thus, tiie count of die zeros may be sent for display without 
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sending the zeros. A 3D image buffer in the MVD controller 18 may be 
initialized to store all zeros, and then as the image data is stored in the image 
buffer, a detection of the ZRF flag causes the MVD controller 18 to jump ahead 
in the buffer by the number of data positions or pixels equal to the run length of 
5 zeros. The 3D data image buffer then contains the 3D data to be output to the 
image projector 20, which may include an SLM driver for operating an SIM to 
generate the two-dimensional images. 

[ 0049] The image projector 20 has associated optics 22 for projecting the 
two-dimensional sUces of 24-30 of the 3D image at a high frame rate and in a 

10 time-sequential manner to a multiple optical element (MOE) device 32 for 
selective imaging to generate a first volumetric three-dimensional image 34 
which appears to the viewer 12 to be present in the space of the MOE device 32. 
The MOE device 32 includes a plurality of optical elements 36-42 which, under 
the control of the MVD controller 18, selectively receive each of the slices 24-30 

15 as displayed two-dimensional images 44-50, with one optical element receiving 
and displaying a respective slice during each frame rate cycle. The number of 
depth slices generated by the MVD controller 18 is to be equal to the number of 
optical elements 36-42, that is, each optical element represents a unit of depth 
resolution of the volumetric 3D image to be generated and displayed. 

20 [ 0050] The optical elements 36-42 may be liquid crystal displays 

composed of, for example, nematic, ferroelecetric, or cholesteric materials, or 
other polymer stabilized materials, such as cholesteric textures using a modified 
Kent State formula known in the art for such compositions. 
[ 0051] The overall display of each of the slices 24-30 by the optical 

25 elements 36-42 of the MOE device 32, as a set of displayed images, occurs at a 
sufficiently high frame rate as set forth below, such as rates greater than about 
35 Hz so that human viewer 12 perceives a continuous volumetric 3D image 34, 
viewed directiy and without a stereographic headset, and instead of die 
individual two-dimensional images 44-50. Accordingly, in the Ulustration of 

30 FIG. 1, the images 44-50 may be cross-sections of a sphere, and so die 3D image 
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34 thus generated which would appear as a sphere to the viewer 12 positioned 
in the midst of the optical elements 36-42 forming the MOE device 32. 
[0052] In alternative embodiments, the images 44-50 may be generated 
to display an overall image having a mixed 2D and 3d appearance, such as 2D 
5 text as a caption below a sphere, or 2D text on the sphere. One application may 
be a graphic user mterface (GUI) conti^l pad which has both 2D and 3D image 
characteristics to allow the viewer 12 to view a GUI, such as MICROSOFT 
WINDOWS 95, with 2D screen appearances as a virtual flat screen display, and 
with 3D images such as the sphere appearing, on a virtual flat screen display. 
10 [ 0053] The first volumetric 3D image 34 is viewable witiiin a range of 
orientations. Furthermore, light 52 from the first volumetric 3D image is further 
processed by a real image projector 54 to generate a second volumetric 3D 
image 56 which appears to tiie viewer 12 to be substantially tiie same image as 
die first volumetric 3D image 34 floating in space at a distance from the MOE 
15 device 32. The real image projector 54, or alternatively a floating image 
projector, may be a set of optics and/or mirrors for coUecting light 52 emitted 
from die MOE device 32 and for re-imaging the 3D' image 34 out into free 
space. The teal image projector 54 may be a high definition volumetric display 
(HDVD) which includes a conventional spherical or parabolic mirror to produce 
20 a signal viewing zone located on an optic axis of the MOE device 32. 
[ 0054] For example, the real image projection systems may be die 
apparatus described in U.S. Patent Nos. 5,552,934 to Prince and 5,572,375 to 
Crabtree, IV, each of these patents being incorporated herein by reference. In 
alternative embodiments, holographic optics may be employed by the real 
25 image projector 54 with die same functions as conventional spherical or 
paraboUc mirrors to generate a floating image 56 but witii multiple viewing 
zones, such as one viewing zone in a center area aligned with the optic axis, and 
viewing zones on eidier side of an optical axis, so multiple 3D floating images 
56 may be viewed by multiple viewers. 
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[ 0055] In other alternative embodiments, the real image projector 54 may 
include holographic optical elements (HOEs), that is, holograms in the 
conventional sense which do not show a recorded image of a pre-existing object 
Instead, an HOE acts as a conventional optical element such as a lens and/or 

5 mirror to receive, reflect, and re-direct incident Ught. Compared to 

conventional optical elements such as glass or plastic, HOEs are very Ughtweight 
and inexpensive to reproduce, and may also possess unique optical 
characteristics not avaUable in conventional optics. For example, an HOE can 
produce multiple images of the same object at different angles from a 
10 predetermined optical axis, and so the field of view of a display employing a 
relatively small HOE can be dramatically increased witiiout increasing the optic 
size as required for conventional optics. Accordingly, using at least one HOE as 
the real image projector 54, the MVD system 10 may be fabricated to provide a 
relatively compact system with a 360' field of view. In addition, for an image 

15 projector 20 incorporating laser Ught sources, HOEs are especially compatible 
for high performance with such laser Ught sources dues to the wavelengtii 
selectivity of the HOE. 

[ 0056] Since either of the volumetiic 3D images 34, 56 appears to the 
viewer 12 to have volume and depth, and optionally also color, the multi-planar 

20 volumetric display system 10 may be adapted for virtual reality and 

haptic/tactile applications, such as the example described below for tactile 
animation to teach surgery. The real image projector 54 allows tixe floating 3D 
image 56 to be directiy accessible for virtual interaction. The MVD system 10 
may include a user feedback device 58 for receiving hand movements from 

25 viewer 12 corresponding to tiie viewer 12 attempting to manipulate ddier of 
die images 34, 56. Such hand movements may be translated by ±e user 
feedback device 58 as conti-ol signal which are conveyed to the interface 14 to 
tiie MVD controUer 18 to modify one or botii of tiie images 34, 56 to appear to 
' respond to the movements of die viewer 12. Alternatively, the user feedback 

30 device 58 may be operatively connected to ±e graphics data source 16, which 
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may include a 3D graphics processor, to modify one or both of the images 34, 
56. 

[ 0057] A nimiber of new interactions technologies provide improved 
performance of the MVD 10 using the real image projector 54. For example, a 
5 force feedback interface developed by SENSIBLE DEVICES, INC. of Cambridge, 
Massachusetts, is a powerful enabling technology which allows the MVD system 
10 to provide the ability to actually feel and manipulate the 3D images 34, 56 
by hand. With appropriate programming, the viewer 12 can sculpt three- 
dimensional images as if the images were clay, using a system called DIGITAL 
10 CLAY, a commercial product of DIMENSIONAL MEDIA ASSOCIATES, the 
assignee of the present application. 

[ 0058] Another application of a MVD system 10 with force feedback 
interface is a surgical simulator and trainer, in which the user can see and feel 
three-dimensional virtual anatomy, including animation such as a virtual heart 
15 beating and reacting to virtual prodding by a user, in order to obtam 

certification as a surgeon, to practice innovative new procedures, or even to 
perform a remote surgery, for example, over the Internet using Intemet 
commimication protocols. 

[ 0059] Tactile effects may thus be combined with animation to provide 
20 real-time simulation and stimulation of users working with 3D images generated 
by the MVD system 10. For example, the viewer 12 may be a surgeon teaching 
medical students, in which the surgeon views and manipulates the first 3D 
image 34 in virtual reality, while the students observer the second 3D image 56 
correspondingly manipulated and modified due to the real image projector 54 
25 responding to changes in the first 3D image 34. The students ±an may take 
turns to individually manipulate the image 34, such as the image of the heart, 
which may even be a beating heart by imaging animation as the 3D images 34, 
54. The teaching surgeon may then observe and grade students in performing 
image manipulation as if such images were real, such as a simulation of heart 
30 surgery. 
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THE MOE DEVICE 

[ 0060] In an illustrated embodiment, the MOE device 32 is composed of a 
stack of single pixel liquid crystal displays (LCDs), composed of glass, as the 
optical elements 36-42, which are separated by either glass, plastic, liquid, or air 
5 spacers. Alternatively, the optical elements 36-42 may be composed of plastic 
or other substances with various advantages, such as lightweight construction. 
The glass, plastic, and/or air spacers may be combined with the glass LCDs in 
an optically continuous configuration to eliminate reflections at internal 
interfaces. The surfaces of the LCDs and spacers may be optically combined by 

10 ei±er optical contact, index matching fluid, or optical cement. Alternatively, 
±e spacers may be replaced by liquid such as water, mineral oil, or index 
matching fluid, with such liquids able to be circulated through an external 
chilling device to cool the MOE device 32. Also, such liquid-spaced MOE 
devices 32 may be transported and installed empty to reduce the overall weight, 

15 and die spacing liquid may be added after installation. 

[ 0061] In a preferred embodiment, tiie optical elements 36-42 are planar 
and rectangular, but alternatively may be curved and/or of any shape, such as 
cylindrical. For example, cylindrical LCD displays may be fabricated by 
difference techniques such as extrusion, and may be nested within each other. 

20 The spacing distance between tiie optical elements 36-42 may be constant, or in 
alternative embodiments may be variable such tiiat the deptii of the MOE device 
32 may be greatiy inaeased without increasing the number of optical elements 
36-42. For example, since the eyes of the viewer 12 lose depth sensitivity witii 
increased viewing distance, the optical elements positioned further from the 

25 viewer 12 may be spaced further apart. Logarithmic spacing may be 
implemented, in which the spacing between the optical elements 36-42 
increased Unearly with the distance from the viewer 12. 
[ 0062] The optical elements 36-42 are composed of a liquid crystal 
formulation vdth the property to be electronicaUy switched rapidly, for example, 

30 by a MOE device driver of the MVD controller 18, to be switched between a 
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clear, highly transparent state, as shown in FIG. 2, and a opaque, highly 
scattering state, as shown in FIG. 3. Referring to FIGS. 2-3 with a cross-section 
of the optical element 36 being illustrated, liquid crystal molecules 60-64 may 
be suspended between the substrates 66-68^ which may be glass, plastic, or air 
5 spacers, and may also have transparent conducting layers 70, 71 applied to 
substrates 66-68, respectively. 

[ 0063] The conducting layers 70, 71 may be composed of a sputtered or 
evaporated thin film of indium tin oxide (ITO), which has an excellent 
transparency and low resistance, but has a relatively high refractive index 

10 compared to the refractive indices of the glass or plastic substrates 66-68. The 
refractive index difference between these materials may produce reflections at 
the interfaces thereof, so additional coatings or layers of anti-reflection (AR) 
materials may optionally be disposed on the substrates 66-68 between the 
conducting layers 70, 71 and the substrates 66-68 to reduce the amount of 

15 reflected light, such as xmwanted reflections. For example, an AR layer having 
an optical thickness of one quarter of a typical wavelength of light, such as 
76 nm., and having a refractive index of about 1.8 reduces the reflection at the 
substrate-conductive layer interface to very low levels. 
[ 0064] By using the AR coatings, the spacing material between optical 

20 elements 36-42 may be removed to leave air or vacuum therebetween, tiius 
reducing tiie overall weight of die MOE device 32. Such AR coatings may be 
vacumn deposited, or may be evaporated or sputtered dielectrics. Alternatively, 
the AR coatings may be applied by spin coating, dip coating, or meniscus 
coating with SOL-GEL. 

25 [ 0065] Referring to FIG. 2, using such conductive layers 70, 71, a source 
72 of voltage therebetween, for example, from the MVD controller 18, generates 
an electric field 74 between the substrates 66-68 of the optical element 36, 
which causes liquid crystal molecules 60-64 to align and to transmit light 76 
through the optical element 36 with litde or no scattering, and so the optical 

30 element 36 is substantially transparent. 
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[ 0066] Referring to FIG. 3, removal of the voltage 72 may occur, for 
example, by opening the circuit between the conductive layers 70, 71, such as 
by opening a rapidly switchable switch 78 controlled by the MVD controller 18. 
Upon such a removal of the voltage 72, the liquid crystal molecules 60-64 are 
5 oriented randomly, and so Ught 76 is randomly scattered to generate scattered 
light 80. In this configuration, the optical element 36 appears opaque, and so 
may serve as a projection screen to receive and display the respective image 44 
focused thereupon by ±e image projector 20. 
[ 0067] In an alternative embodiment, referring to FIGS. 2-3, the 
10 illustrated optical element 36 may be activated to be in the transparent state 
shown in FIG.2 by connecting the conductive layer 70 adjacent to a first 
substrate 66 to ground while connecting the conductive layer 71 adjacent to a 
second substrate 68 to a supply voltage, such as a voltage in the range of about 
50 V to about 250 V. To switch the optical element 36 to be in die scattering, 
15 opaque state as in FIG. 3, die application of voltage is reversed, tiiat is, the 
conductive layer 71 is grounded for a predetermined delay such as 1 ms to 
about 5 ms, and then the conductive layer 70 is connected to the supply voltage. 
The procedure is again reversed to return the optical element 36 to the 
transparent state. Accordingly, no average direct current (DC) or voltage is 
20 applied to ±e optical element 36, which can lead to failure by having a constant 
applied voltage. Also, ±ere is no continuous alternating current (AC) or voltage 
which generates heating and increases power requirements to the optical 
elements. 

[ 0068] In operation, only a single one of the optical elements 36-42 of the 
25 MOE device 32 is in the scattering opaque state at any given time, thus forming 
a scattering plane or surface. As the image projector 20 projects die sUces 24-30 
at a high rate through a projection cycle, with one slice emitted per cycle, the 
scattering plane is rapidly rastered through die depdi of the MOE device 32 to 
form an weekly variable depth projection screen, while the remaining 
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transparent optical elements permit the viewer 12 to see the displayed image 
irom the received image slices 24-30. 

[ 0069] As shown in FIGS. 4-7, as successive frame data is fed from the 
MVD controller 18 to the image projector 20 to generate images 82-88 

5 therefrom, the MVD controner 18 synchronizes the switching of the optical 
elements 36-42 such that the optical 36 is opaque as the image 82 is emitted 
thereon as in FIG. 4, the optical element 38 is opaque as the image 84 is emitted 
in FIG. 5, the optical element 40 is opaque as the image 84 is as in FIG. 6, and 
tiie optical element 42 is opaque as the image 88 is emitted thereon as in FIG. 7. 

10 The MVD controller 18 may introduce a delay between feeding each set of 
frame data to tiie image projector 20 and causing a given optical element to be 
opaque so that the image projector 20 has enough time during the delay to 
generate the respective images 82-88 from the sets of frame data 1-4, 
respectively. 

15 [ 0070] Referring to FIGS. 4-7, while one optical element is opaque and 
displays tiie respective image thereon, tiie remaining optical elements are 
transparent, and so the image 82 in FIG. 4 on optical element 36 is visible 
tiirough, for example, at least optical element 38, and simUarly image 84 is 
visible tiirough at least optical element 40 in FIG. 5. and image 86 is visible 

20 tiirough at least optical element 42. Since tiie images 82-88 are displayed at a 
high rate by tiiat image projector 20 onto the optical elements 36-42 which are 
switched to opaque and transparent states at a comparably high rate, tiie images 
82-88 form a single volumetric 3D image 34. 

[ 0071] To form a continuous volumetric 3D image 34 witiiout perceivable 
25 flicker, each optical elements 36-42 is to receive a respective image and is to be 
switched to an opaque state at a frame rate greater tiian about 35 Hz. 
Accordingly, to refresh and/or update die entire 3D image, tiie frame rate of the 
image projector 20 is to be greater tiian about N x 35 Hz. For a stack of 50 LCD 
elements formmg tiie MOE device 32 having an mdividual optical element 
30 frame rate of 40 Hz, ti^ie overall frame rate of tiie image projector 20 is to be 
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greater than about 50 x 40 Hz = 2 kHz. High performance and/or high quality 
volumetric 3D imaging by the MVD system 10 may require greater frame rates 
of the image projector 20 on the order of 15 kHz. 

[ 0072] In one embodiment, the images 82-84 of FIGS. 4-7 are displayed 
5 sequentially, with such sequential frame ordering being the updating of the 
range of depth once per volume period to update the entire volume of optical 
elements 36-42 in the MOE device 32. Such sequential frame ordering may be 
sufficient m marginal frame rate conditions, such as frame displays rates of 
about 32 Hz for still images 82-88 and about 45 Hz for images 82-88 displaying 

10 motion. In an alternative embodiment, semi-random plane ordering may be 
performed to lower image jitter and to reduce motion artifacts, in which the 
range of depth is updated at a higher frequency although each optical element is 
still only updated once per volume period. Such semi-random plane ordering 
includes multi-planar interlacing in which even numbered planes are 

15 illuminated with images, and then odd numbered planes illuminated, which 
increases the perceived volume rate without increasing the frame rate of the 
hnage projector 20. 

[ 0073] The MOE device 32 maintains the image resolution originally 
generated in the image projector 20 to provide high fidelity three-dimensional 

20 images. The liquid crystal panels 36-42 are highly transparent and haze-free in 
the clear, transparent state, and are capable of switching rapidly between the 
clear, transparent state and the opaque, scattering states, in which the light and 
images from ±e image projector 20 is effidentiy and substantially scattered. 
[ 0074] In additional embodiments, the MOE device 32 may be 

25 constructed to be lightweight. The liquid crystal panels 35-42 may be composed 
of a pair of glass substrates coated on ±eir inner surfaces, with the transparent 
conducting layers 70, 71 being coated with an insulating layer. A pol)nner 
alignment layer may optimally be disposed upon the insulating layer. Between 
the substrates of a given liquid crystal panel, a thin layer of liquid crystal 

30 composition is disposed to be about 10-20 microns thick. 
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[ 0075] The majority of the volume and weight of the panels is associated 
with the glass of the substrates, which contributes to a potentially very heavy 
MOE device 32 as the transverse size and the number of panels are increased. 
Implementation of the liquid crystal panels 36-42 to be composed of plastic 
5 substrates is one solution to the increase in weight. Other implementations 
include using processing methods to produce the optical elements of the MOE 
device 32 on a roll-to-roll process on very thin plastic substrates, to allow 
febrication to be produced by a continuous and very low cost me±od. 
[ 0076] Using such relatively lightweight components for the MOE device 
10 32, the Moe device 32 may also be collapsible when not in operation, to allow 
the MVD system 10 to be portable. Also, the optical elements 36-42 may 
include other inorganic materials in addition to or instead of liquid crystal 
technology, such as an ITO layer organically applied by spin or dip coating. 

THE HIGH FRAME RATE IMAGE PROJECTOR 
15 [ 0077] The maximum resolution and color depth of the three-dimensional 
images 34, 56 generated by the MVD system 10 is direcdy determined by the 
resolution and color dep± of the high frame rate image projector 20. The role 
of the MOE device 32 is primarily to convert the series of two-dimensional 
images from the image projector 20 into a 3D volume image. 
20 [ 0078] In one embodiment, the image projector 20 includes, an arc lamp 
light source with a short arc. The light from the lamp is separated into red, 
green and blue components by color separation optics, and is used to illuminate 
three separate spatial light modulations (SLMs). After modulation by the SLMs, 
the three color channels are recombined into a single beam and projected from 
25 the optics 22, such as a focusing lens, into the MOE device 32, such that each 
respective two-dunensional image from the slices 24-30 is displayed on a 
respective one the optical elements 36-42. 

[ 0079] In another embodiment, the image projector 20 includes high 
power solid state lasers instead of an arc lamp and color separation optics. 
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Laser light sources have a ntimber of advantages, including, increased 
efficiency, a highly directional beam, and single wavelength operation. 
Additionally, laser light sources produce highly saturated, bright colors. 
[ 0080] In a fiirther embodiment, different technologies may be used to 
5 implement the SIM, provided that high speed operation is attained. For 
example high speed Uquid crystal devices, modulations based on micro- 
electromechanical (MEMS) devices, or otiier light modulating metiiod may be 
used to provide such high frame rate imaging. For example, the Digital Ught 
Processing (DLP) technology of TEXAS INSTRUMENTS, located in Dallas, 

10 Texas; the Grating Light Valve (GLV) technology of SILICON UGHT 

MACHINES, located in Sunnyvale, California; and the analog ferroelectiic LCD 
devices of BOULDER NONLINEAR SYSTEMS, located in Boulder, Colorado, may 
be used to modulate the images for output by the image projector 20. Also, the 
SLM may be a ferroelectric liquid crystal (FLC) device, and polarization biasmg 

15 of the FLC SLM may be implemented. 

[ 0081] To obtain very high resolution images in the MVD system 10, the 
images 44-50 must be appropriately and rapidly re-focused onto each 
corresponding optical element of die MOE device 32, in order to display each 
corresponding image on the optical element at the at tiie appropriate deptii. To 

20 meet such re-focusing requirements, adaptive optics systems are used, which 
may be device known in the art, such as the fast fbcusmg apparatus described in 
G. Vdovin, "Fast focusing of imaging optics using micro machined adaptive 
mirrors", available on die Internet at 

hii p!//^emspv-et.tudelft.n 1/fnrns/index.html. As shown in FIG. 8, a 
25 membrane light modulator (MLM) 90 has as a thin flexible membrane 92 which 
acts as a mirror witii contiroUable reflective and focusing characteristics. The 
membrane 92 may be composed of a plastic, nitroceUulose "MYLAR", or then 
metal fihns under tension and coated with a conductive reflecting layer of metal 
coating which is reflective, such as aluminum. An electrode and/ or a 
30 piezoelectiic actuator 94 is positioned to be substantially adjacent to tiie 
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membrane 92. The electrode 94 may be flat or substantially planar to extend in 
two dimensions relative to the surface of the membrane 92. The membrane 92 
is mounted substantially adjacent to the electrode 94 by a mounting structure 
96, such as an eUiptical mounting ring, such as a circular ring. 

5 [ 0082] The electrode 94 is capable of being placed at a high voltage, such 
as about 1,000 volts, from a voltage source 98. The voltage may be varied 
withm a desired range to attract and/or repel the membrane 92. The 
membrane 92, which may be at ground potential by connection to ground 100, 
is this caused by electrostatic attraction to deflect and deform into a curved 

10 shape, such as a parabolic shape. When so deformed, the membrane 92 acts as 
a focusing optic with a focal length and thus a projection distance which can be 
rapidly varied.by varying ±e elecO:ode voltage. For example, the curved surface 
of the membrane 92 may have a focal length equal to half of tiie radius of 
curvature of the curve membrane 92, witii the radius of curvature being 

15 determined by the tension on the membrane 92, the mechanical properties of 
the material of the membrane 92, the separation of the membrane 92 and the 
electrode 94, and the voltage applied to the electirode 94. 
[ 0083] hi one embodiment, the deflection of the membrane 92 is always 
toward the electrode 94. Alternatively, by placing a window with a transparent 

20 conducting layer on the opposite side of the membrane 92 from die electrode 
94, and tiien applying a fbced voltage to the window, the membrane 92 may be 
caused to deflect ui both directions; that is, either away from or toward the 
electrode 94, thus permitting a greater range of focusing images. Such 
controlled variation of such a membrane 92 in multiple directions is described, 

25 for example, in a paper by Martin Yellin in the SPIE CONFERENCE 
PROCEEDINGS, VOL. 75, pp. 97-102 (1976). 

[ 0084] The optical effects of die deflections of tiie MLM 90 may be 
magnified by the projection optics 22, and cause die projected image from an 
object plane to be focused at varying distances from the image projector 20 at 
30 high re-focusmg rates. AdditionaUy, die MLM 90 can maintain a nearly 
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constant image magnification over its full focusing range. 
[ 0085] Referring to FIG. 9, the MLM 90 may be incorporated into an 
adaptive optics system 102. for example, to be adjacent to a quarter wave plate 
104 and a beam spUtter 106 for focusing images to the projection optics 22 

5 Images 110 from an object or object plane 112 pass through the polarizer 108 
to be horizontally polarized by the beam spUtter 106, and thence to pass 
through the quarter wave plane 104 to result in circularly polarized Ught 
incident on the membrane 92 for reflection and focusing. After reflection, such 
focused image 114 are passed back through the quarter wave plate 104 

10 resulting in light 114 polarized at 90- to the direction of the incident light 110. 

The beam spUtter 106 then reflects die light 114 toward the projection optics 22 
to form an image of the object. By using the quarter waveplate 104 and 
polarizer 108 with the MLM 90, the adaptive optic system may be folded into a 
relatively compact configuration, which avoids mounting ±e MLM 90 off-axis 

15 and/or at a distance from the projection lens 22. 

t 00861 The images may be focused at a normal distance F^, to a normal 
projection plane 116 from the projection optics 22, and die image may be 
refocused at a high rate between a minimum distance Fmin from minimum 
projection plane 118 to a maximum distance F„« to a maximum projection * 

20 plane 120 from the projection optics 22 witii high resolution of die image being 
maintained. 

[ 0087] As shown in FIG. 10, die image projector 20 including die 
adaptive optics system with die MLM 90, quarter waveplate 104, and polarizer 
108 may thus selectively and rapidly project individual 2D sUces of the 3D 
25 image onto individual optical elements 36-42, such diat die 2D slices are 

focused on at least one optical element, witii a high focusing accuracy such diat 
die 2D slices are not incident on die spacers 122 between die optical elements 
36-44 of the MOE device 32. 

[ 0088] Referring to FIGS. 9-10, in anodier alternative embodiment, die 
30 image projector 20 may include an SLM 124 having a plurality of pixels 126 for 
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modulating the Ught 110 from the object plane 112. Twisted nematic (TN) 
SLMs may be used, in which a switchable half waveplate is formed by producing 
alignment layers on the front and rear substrates of the SIM 124 which differ in 
orientation by 90*. The liquid crystal of the TN SLM aligns to the alignment 
5 layer on each surface, and then joins smoothly between the substrates to form a 
one-half period of a helix. If the pitch of the helix is chosen to be near the 
wavelength of light, the helix acts as a half-waveplate and rotates the hiddent 
light polarization by 90*. The appUcation of an electric field of sufficient 
strengdi to the TN SLM causes the bulk of the Uquid crystal material between 

10 the two substrates to reorient to point perpendicular to the substrates, which 
unwinds the heUx and destroys the half waveplate, thus eUminating the rotation 
of the polarization of the incident light. The lack of an inherent polarization in 
the TN liquid crystal material causes TN SLMs to be insensitive to the sign of tiie 
applied voltage, and eitiier sign of voltage results in die same reduction in 

15 waveplate action, so the TN SLM acts as waveplate witii a retardation being a 
function of the magnitude of the applied voltage. 
[ 0089] Alternatively, as shown in FIG.ll, die SLM 124 may be 
ferroelectric liquid crystal (FLQ based device composed of a plurality of pbcels 
126, with each pixel 126 having die FLC material 128 positioned over a 

20 semiconductor substrate such as a silicon substrate 130, with an electrode 132 
disposed tiierebetween. The electrode 132 may be composed of aluminum. A 
transparent conductor 134 is disposed above die FLC material 128 and is 
connected to a voltage source, such as a 2.5 V operating voltage. A cover slide 
136 composed, for example, of glass is positioned over the transparent 

25 conductor 134. 

[ 0090] FLC SLMs composed of such pixels 126 operate in a manner 
similar to twisted nematic (TN) SLMs, in which the application of an electric 
field, for example, between die electrode 128 and the conductor 134, results in 
die rotation of polarization of incident light. The degree of rotation is 

30 proportional to the applied voltage, and varies from 0° to 90'. In combination 
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with an external polarizer, such as the polarize 108, the polarization rotation of 
the SLM 124 res\ilts in intensity modulation of the incident light. 
[ 0091] UnUke a TN SLM, an FLC SLM possesses an inherent polarization, 
which results in an FLC SLM having a desired thickness forms a waveplate with 
5 a retardation independent of the appUed voltage. The FLC SLM acts as a 
waveplate with an orientation being a function of both the magnitude and the 
sign of tiie applied voltage. 

[ 0092] For the pixel 126 of the FLC SLM 124 FIG. 11, a half waveplate of 
the FLC SLM 124 is typically implemented to have an unpowered orientation 
10 that is about 22.5° to a horizontal reference axis, resulting in a 45* rotation of 
the incident light polarization. When powered, the transparent conductor 134 
is biased to 2.5 V, which may be half die voltage range of the electrode 132 of 
die pbcel 126. 

[ 0093] Referring to FIGS. 12-14, the orientations of the principle axes of 
15 tiie half waveplate formed by the pbcels 126 of tiie FLC SLM 124 are shown at 0 
V, 2.5 V, and 5 V, respectively, to have a 0', 45*, and 90' polarization, 
respectively. 

[ 0094] Both TN SLMs and FLC SLMs are to be direct current (DC) 
balanced to maintain correct operation. The appUcation of a continuous DC 

20 electric field to the pixels 126 results in tiie destruction of tiie alignment layers 
on tiie substrates by impurity ion bombardment, which ruins the pbcel 126, To 
prevent such damage, tiie electric field is periodically and/or irregularly 
reversed in sign with a frequency on tiie order of about 100 Hz for TN SLMs, 
and about 1 Hz for FLC SLMs. The lack of sensitivity of die TN SLM to die sign 

25 of die electric field results in die image passing theredirough having a constant 
appearance as die electric field is reversed. However, an FLC SLM is typically 
sensitive to die sign of die field, which results in grayscale inversion by which 
black areas of die image changing to white and white areas changing to black as 
the SLM is DC balanced. 
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[ 0095] To prevent grayscale inversion during DC balancing of the SLM 
124, the polarization of the incident light biased so that the positive and 
negative images caused by the appUcation of the electric field to the pixels 126 
have the same appearance. The SLM 124 and/or the individual pixels 126 have 

5 a static half waveplate 138 positioned to receive the incident Ught 110 before 
the SLM 124. The waveplate 138 is oriented to provide a 22.5* rotation of the 
polarization of the incident Ught, with the resulting grayscale having a 
maximum brightness with either 0 V or 5 V are appUed to the electrode 132, 
and has a minimum brightoess when 2.5 V is appUed to the electrode 132. In 

10 alternative embodiments, to prevent reduction of the maximum brightoess by 
inclusion of the waveplate 138, FLC material 128 having a static orientation of 
45° may be used, which allows the maximum brightness of a polarization biased 
FLC SLM 124 to match the maximum brightoess of the unbiased SLM without 
the waveplate 138. 

15 [ 0096] As described above, in alternative embodiments of the image 
projector 20, lasers may be used such as colored and/or solid state color- 
producing lasers at the object plane 112. Such lasers may, for example, 
incorporate blue and green soUd state lasers currently available in other 
information storage and retrieval technologies, such as CDROMs as weU as laser 

20 video systems. 

[ 0097] In one alternative embodiment of the image projector 20, the 
adaptive optics may be used in a heads-up display to product the 3D image that 
is not used in depth but instead may be moved toward or away from the viewer 
12. Without using the MOE device 32, the 2D image sUces 24-30 may be 

25 projected direcdy into the eye of the viewer 12 to appear at the correct depth. 
By rapidly displaying such slices 24-30 to the viewer 12, a 3D image is 
perceived by the viewer 12. In diis embodiment of the MVD system 10, the 
adaptive optics of the image projector 20 and other components may be very 
compact to be incorporated into existing heads-up displays for helmet-mounted 

30 displays or in cockpit or dashboard mounted systems in vehicles. 
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[ 0098] In another embodiment, the sUces 24-30 may be generated and 
projected such that some of the images 44-50 are respectively displayed on 
more than one of optical elements 36-42, in order to oversample the depth by 
displaying the images over a range of depths in the MOE device 32 instead of at 

5 a single depth corresponding to a single optical element. For example, 

oversampling may be advantageous if the MOE device 32 has more planes of 
optical elements 36-42 tiian the number of image slices 24-30, and so tiie 
number of images 44-50 is greater than tiie number of image slices 24-30. For 
example, a slice 24 displayed on both of optical elements 36-38 as images 44- 

10 46, respectively. Such oversampHng generates the 3D image 34 witii a more 
continuous appearance witiiout increasing tiie number of optical elements 36- 
42 or die frame rate of tiie image projector 20. Such oversampling may be 
performed, for example, by switching multiple optical elements to be in an 
opaque state to receive a single projected slice during a respective multiple 

15 projection cycles onto the respectively opaque multiple optical elements. 

GENERATION OF THE 3D IMAGE FROM A MULTI-PLANAR DATASET 
[ 0099] To generate tiie set of 2D image sUces 24-30 to be displayed as a 
set of 2D images 44-50 to form die 3D hnage 34, a multi-planar dataset is 
generated from the 3D image data received by tiie MVD controller 18 from tiie 

20 graphics data source 16. Each of tiie sUces 24-30 is displayed at an appropriate 
deptii witiiin tiie MOE device 32; tiiat is, die slices 24-30 are selectively 
projected onto a specific one of tiie optical elements 36-42. If tiie sUces 24-30 
of the 3D image 34 are made close enough, tiie image 34 appears to be a 
continuous 3D image. Optional multi-planar anti-aliasing described hereto may 

25 also be employed to enhance tiie continuous appearance of tiie 3D image 34. 
[ 0100] A metiiod of computing a multi-planar dataset (MPD) is 
performed by tiie MVD system 10. In particular, tiie MVD controller 18 
performs such a metiiod to combine die information from a color buffer and a 
deptii (or z) buffer of tiie frame buffer of tiie graphics data source 16, which 
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may be a graphics computer. The method also includes fixed depth operation 
and anti-aliasing. 

[ 0101] Referring to FIG. 15, the method responds in step 140 to 
interaction wi± the user 12 operating the MVD system 10, such as through a 
5 GUI or the optional user feedback device 58 to select and/or manipulate the 
images to be displayed. From such operation and/or interaction, the MVD 
system 10 performs fanage rendering in step 142 fi-om image data stored in a 
frame buffer, which may be, for example, a memory of the MVD controller 18. 
The frame buffer may include sub-buffers, such as the color buffer and the 

10 depth buffer. During a typical rendering process, a graphics computer computes 
the color and depth of each pixel in the same (x,y) position in the depth buffer. 
If the depth of the a new pixel is less than the depth of the previously computed 
pixel, then the new pixel is closer to the viewer, so the color and depth of the 
new pixel are substituted for the color and depth of the old pixel in both of the 

15 color and depth buffers, respectively. Once all objects in a scene are rendered 
as a dataset for imaging, the method continues in steps 144-152. Alternatively 
or addition, ±e rendered images in ±e frame buffer may be displayed to the 
viewer 12 as a 3D image on a 2D computer screen as a prelude to generation of 
die 3D image as a volumetric 3D image 34, thus allowing the viewer 12 to select 

20 which images to generate as the 3D image 34. 

[ 0102] In performing the method for MPD computation, the data from the 
color buffer is read in step 144, and the data from ±e depth buffer is read in 
step 146. The frame buffer may have, for example, tiie same number of pbcels 
in the x-dimension and the y-dimension as the desired size of the image slices 

25 24-30, which may be determined by die pixel dimensions of the optical 

elements 36-42. If the number of pixels per dimension is not identical between 
the frame buffer and the image slices 24-30, the data in the color and depth 
buffers are scaled in step 148 to have the same resolution as die MVD system 10 
with the desired pixel dimensions of the image slices 24-30. The MVD 

30 controller 18 includes an output buffer in the memory for storing a final MPD 
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generatfid from the data of the color and depth buffers, which may be scaled 
data as indicated above. 

[ 0103] The output buffer stores a set of data corresponding to the 2D 
images, with such 2D images having the same resolution and color depth as the 
5 images 44-50 to be projected by the sUces 24-30. In a preferred embodunent, 
the number of images 44-50 equals the number of planes formed by the optical 
elements 36-42 of tiie MOE device 32. After tiie MPD calculations are 
completed and the pixels of tiie 2D images are sorted in tiie output buffer m 
step 150, die output buffer is transferred to an MVD image buffer, which may 
10 be maintained in a memory in tiie image projector 20, from which tiie 2D 
images are converted to image sUces 24-30 to form tiie 3D image 34 to be 
viewed by ±e viewer 12, as described above. The metiiod tiien loops back to 
step 140, for example, concurrentiy with generation of the 3b image 34, to 
process new inputs and tiience to update or change tiie 3D image 34 to 
15 generate, for example, animated 3D images. 

[ 0104] The MVD system 10 may operate in two modes: variable deptii 
mode and fixed depth mode. In variable deptii mode, tiie deptii buffer is tested 
prior to tiie MPD computations including step 146, in order to determine a 
maximum deptii value Zmax and the minimum deptii value Z^^,, which may 
20 correspond to die extreme deptii values of flie 3D image on a separate 2D 
screen prior to 3D volumetric imaging by tiie MVD system 10. In die fixed 
deptii mode, die and Zmin are assigned values to ±e viewer 12, eidier 
interactively or during appUcation startup to indicate tiie rear and firont bounds, 
respectively, of the 3D image 34 generated by tiie MVD system 10. Variable 
25 deptii mode allows all of die objects visible on tiie 2D screen to be displayed in 
tiie MOE device 32 regardless of tiie range of deptiis or of changes in image 
depth due to interactive manipulations of a scene having such objects. 
[ 0105] In fixed deptii mode, objects which may be visible on tiie 2D 
SCTeen may not be visible in tiie MOE device 32 since such objects may be 
30 outside of a virtual depdi range of die MOE device 32. In an alternative 
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embodiment of the fixed depth mode, image pixels which may be determined to 
lie beyond the "back" or rearmost optical element of the MOE device 32, relative 
to the viewer 12, may instead be displayed on the rearmost optical element. For 
example, from the perspective of the viewer 12 in FIG. 1, the optical element 36 
5 is the rearmost optical element upon which distant images may be projected. In 
this manner, the entire scene of objects remains visible, but only objects with 
depths between and Z^^ are visible in the volumetric 3D image generated 
by the MOE device 32. 

[ 0106] In the MPD method described herein, using the values of Zmax and 
10 Z.„ the depth values within the depth buffer may be offset and scaled in step 
148 so that a pixel wi± a depth of Z„„ has a scaled depth of 0. and a pixel with 
depth of has a scaled depth equal to the number of planes of optical 
elements 36-42 of the MOE device 32. In step 150, such pixels with scaled 
depths are then sorted and stored in ±e output buffer by testing the integer 
15 portion [dj of die scaled depth values d,. and by assigning a color value from 
the color buffer to the appropriate MPD sUces 24-30 at the same (x,y) 
coordinates. The color value may indicate tiie brightness of the associated pkel 
or voxel. 

[ 0107] Based on the foregoing, it will be evident to one sldlled in the art 
20 that the same effects can be achieved by using a selected subset of the optical 
elements 36-42 of MOE device 32. However, in tiie preferred embodiment all 
optical elements 36-42 of MOE device 32 are utilized. 
[ 0108] Keeping in mind the distinction between voxels and tridels, as 
discussed above, tiie process of mapping die deptii of a tiidel from virtual space 

25 to its voxel deptii coordinate witiiin tiie display actually occurs in two steps. 
The first step entails conversion of tiie virtual deptii-coordinate (z) of the tridel 
into an actual deptii coordinate (z' ) witiiin tiie multiplanar display. The second 
step entaUs converting the continuous z' values of tiie tiidel to tiie discrete deptii 
coordinate k of a particular display voxel (k). The reasons for tiiis will become 

30 apparent below. 
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[ 0109] The conversion from z to z' can be carried out in either the MVD 
controller 18 or in graphics data source 16. Since this conversion is somewhat 
display independent it is preferably carried out by software (either application, 
API, or device driver) or graphics card hardware within the MVD controller 18. 
5 Similarly the conversion from z' to k can be carried out either in the MVD 
controller 18 or graphics data source 16. However, since this conversion 
depends on the specific parameters of the display it will often be carried out in 
the MVD controller 18, either by hardware or firmware. 
[ 0110] However, in systems in which the multiplanar frame buffer is 
10 actually on a graphics card of the graphics data source 16, the conversion from 
z* to k must be carried out in the graphics card hardware. In this case, the 
graphics card must be able to query the MVD controller 18 as to its z' to k 
mapping characteristics so that these may be used during the processing of 
tridels into voxels. 

15 [ 0111] The virtual depth coordinate within the graphics data source 16 
can potentially have a range that is much deeper that the physical depth of the 
volumetric display. For example, a scene of a house and street can have a 
virtual depth range of a 50 meters, whereas the MOE device 32 may be 
physically only 0.3 meters deep. Further, the mapping of a tridePs virtual depth 

20 z to physical depth z* may take any functional form provided it is a single 

valued. For example, in the variable depth mode discussed above, the simplest 
mapping is to scale the entire virtual depth range Dy to fit linearly witfiin the 
depth Dj5 of MOE device 32 with a constant scale parameter equal to D^Dy. 
Similarly, in the fixed depth mode discussed above, the first 0.3 meters of the 
. 25 virtual space could be mapped to the display with a constant scale of 1. The 
parts of the scene with depth greater than Dd can be either not displayed, or be 
painted onto the deepest plane of the display as a 2d backdrop. 
[ 0112] Another useful mapping might be one that is nonlinear and 
provides high resolution for low depth values and reduced resolution at higher 

30 depth values. For example, the square root function provides the highest 
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resolution near zero with decreasing resolution as z increases. An example 
using the preceding values for Dy and is in to use the mapping: 

z' = Dd-Jz = 0.0424VI 

5 for z in the range of 0 to 50 meters. In general any single valued function can 
be used to map z to z' and it will be left to the programmer or viewer to decide 
how to make the most appropriate z to z* mapping for the particular image or 
application. 

[ 0113] In order to create an image within the MOE device 32 a method is 
10 required to compute the discrete voxel depth k from the desired physical depth 

a of the tridel The MOE device 32 is composed of a number of optical elements 

or unage planes (Npi^nes) that occupy a range of physical depths between 0 and 
In the simplest case the planes can be equally spaced by an amount a= 

Dr/CNpianes TMs makes the relationship between z' and k simple, linear and 
15 equal to k = z'/a. However, it may be sometimes desirable, to have ±e spacing 

between planes increase with increasing depth from the viewer. In tiiis case the 

relationship between z* and k becomes nonlinear. For example, if the spacing 

between planes k and k + 1 is given by: 

' 20 then the overall depth of the display is 

n (m iU , (^Flan^-T^i^ Flees 

and the physical depth z' of plane k is 



X73357.1 



wo 03/017246 



PCT/OS02/26718 



-33- 

[ 0114] The above equation can be solved for k to give 



1 A r i A,^ 2AiZ'^ 

2 V ^0 4Ao Ao J 

[ 0115] By inspection we can determine that the positive root of the above 
equation is the one to use to compute the voxel depth k irom the physical depdi 

5 z' since the negative root would give negative value, a clearly nonphysical 
solution. Although the voxel depth could be computed from the above equation 
"on the ny" as voxel data is transferred to the display, it may be more efficient 
to use a pre-computed lookup table since the range of both z' and k will be 
known from the design of the MOE device 32. 

10 [ 0116] It will be noted that die above equation does not, in general, give 
an integer value as a result. This is acceptable because multiplanar anti-aUasing 
serves to determine how the brightness of a voxel at depth k associated with a 
tridel at virtual depth z can be divided among two adjacent display voxels. 
Recall that the integer part of k determines the pair of planes to which tiie 

15 brighmess of a tridel is assigned and the fractional part of k determines how the 
brighmess is apportioned between the two planes. For example, if a tridel at (i, 
j) has a value of k equal to 5.34, tiien 34% of die tridel's brightness will be 
found on die voxel at (ij,6) of die tridel's brighmess wall be found on die voxel 
at (ij,5). 

20 [ 0117] Using the disclosed MPD mediod, die volumetric 3D unages 34 
generated by tiie MVD system 10 may be incomplete; tiiat is, objects or portions 
diereof are completely eliminated if such objects or portions are not visible from 
die point of view of a viewer viewing die corresponding 3D image on a 2D 
computer screen. In a volumetiic display generated by die MVD system 10, 

25 image lookaround is provided allowing die viewer 12 in FIG. 1 to move to an . 
angle of view such that die previously hidden objects become visible, and so 
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such MVD systems 10 are advantageous over existing 2D displays of 3D images. 
[ 0118] In alternative embodiments, the MPD method may implement 
anti-aUasuxg, as described herein, by using fractional portion of scaled depth 
value; tiiat is, d, - M to assign such a fraction of the color value of the pixels to 
5 two adjacent MVD image sUces in the set of sUces 24-30. For example, if a 
scaled depth value U 5.5 and each sUce conresponds to a discrete depth value, 
half of the brightness of tiie pixel is assigned to each of slice 5 and sUce 6. 
Alternatively, if tiie scaled deptii is 5.25, 75% of ±e color value is assigned to 
sUce 5 because slice 5 is "closer" to die scaled deptii, and 25% of tiie color value 

10 is assigned to slice 6. 

[ 01 19] Different degrees of anti-aliasing may be appropriate to different 
visualization tasks. The degree of anti-aliasing can be varied from one extreme; 
tiiat is, ignoring tiie fractional depth value to assign tiie color value, to anotiier 
extreme of using all of tiie fractional deptii value, or tiie degree of anti-aliasing 
15 can be varied to any value between such extremes. Such variable anti-aUasing 
may be performed by multiplying tiie fractional portion of tiie scaled deptii by 
an anti-aliasing parameter, and tiien negatively offsetting tiie resulting value by 
half of tiie anti-aliasing parameter. The final color value may be detennined by 
fixing or clamping tiie negatively offset value to be widiin a predetennined 
20 range, such as between 0 and 1. An anti-aUasing parameter of 1 corresponds to 
full anti-aUasing, and an anti-aliasing parameter of infinity corresponds to no 
anti-aliasing. Anti-aliasing parameters less tiian 1 may also be implemented. 
[ 0120] In scaUng depth buffer values, a perspective projection may be 
used, as specified in tiie Open Graphics Ubrary (OpenGL) multi-platform 
25 software interface to graphics hardware supporting rendering and imaging 

operations. Such a perspective projection may result in a non-linearity of values 
in ±e deptii buffer. For an accurate relationship between tiie virtual deptii and 
tiie visual deptii of die 3D image 34, tiie MVD controller 18 takes such non- 
linearity into account to scale tiie deptii buffer values in step 148. Alternatively, 
30 an ortiiogtaphic projection may be used to scale die deptii buffer values in step 
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148. 

[ 0121] It win be appreciated by those sWUed in the art that there are 
many factors that contribute to the abiUty of human vision to perceive objects or 
scenes in three-dimensions. Among these fectors are both physical vision cues 

5 and psychological vision cues. By vvay of example, physical vision cues arise 
from, but are not Umited to, the foUowing physical effects. 
[ 0122] Three dimensionaUty of a scene is associated with the fact that 
slighdy different images are provided to each eye. This binocular effect or so- 
called stereopsis. is an important physical cue that is processed by the brain to 

10 impart three-dimensionality to what is being viewed. Further, in viewing a real 
three-dimensional scene, the viewer's eyes must change their focus as they focus 
to different depths within the three-dimensional scene. This difference in eye 
focusing, sometimes referred to as eye accommodation, is another physical 
vision cue that permits the brain to conclude that a three-dimensional scene is 

15 being viewed. A closely related physical cue is ocular convergence, which 

means that both eyes must point toward and focus on the same spot. In viewing 
a real three-dimensional scene, the amount of ocular convergence varies as die 
eye focuses on different depths within the three-dimensional scene. This 
provides another physical cue to die brain that the scene being vievred is three 

20 dimensional. 

[ 0123] Another example of a physical cue arises from the fact that a real 
three-dimensional scene requires movement of the observer to view different 
portions of the three-dimensional scene. This so-called "image look around" or 
motion parallax is yet another physical cue associated with real three- 
25 dimensional scenes which imparts to the brain the perception tiiat a viewed 
scene is indeed three-dimensional. 

[ 0124] Physical vision cues, as exemplified by the above effects, are 
inherendy present in the volumetric three-dimensional images disclosed herein 
because they are created in and occupy a volume of space. These physical cues 
30 distinguish such images from unages tiiat appear to be three-dimensional but 
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are in fact rendered on a two-dimensional display such as a television screen or 
computer monitor. 

[ 0125] By their very nature, the volumetric three-dimensional image 
displays disclosed herein produce images having a measurable but finite depth. 

5 While this depth can be adjusted by varying the geometry of the MOE device 32, 
including the number and spacing of the plurality of optical elements 36-42 
contained therein, the perceived deptii of volumetric images produced by the 
MOE 32 is necessarily limited by practical considerations. 
[ 0126] It is known in the art tiiat in addition to tiie physical vision cues 

10 provided to the brain when viewing real tiiree-dimensional scenes, it is also 
possible to create and emphasize the illusion of depth or three-dimensionality 
witiiin a two-dimensional image by the use of one or more psychological cues. 
By way of example, and not limitation, psychological vision cues may be 
provided by rendering a scene with appropriate shading and/or shadowing to 

15 give objects in the scene the appearance of deptii to tiiereby unpart an overaU 
three dimensional appearance to the scene. 
[ 0127] A common psychological vision cue is the use of forced 
perspective. In existing 2D monitors, perspective is generated computationally 
in tiie visualization of 3D data to create a sense of depth such that objects 

20 furtiier from the viewer appear smaller, and parallel lines appear to converge. 
In tiie disclosed MVD system 10, tiie 3D image 34 is generated witii a 
computational perspective to creative tiie aforesaid sense of deptii, and so tiie 
depth of the 3D image 34 is enhanced. 

[ 0128] Further, a scene may be provided with a three-dimensional 
25 appearance by rendering objects witiiin tiiat scene so tiiat tiiey have a surface 
texture whose resolution decreases widi apparent distance of tiie objects from 
tiie viewer. This provides a "fuzziness" to die appearance of surfaces which 
increases as dieir apparent deptii witiiin tiie scene increases. Gosely related to 
tills psychological vision cue is the addition of atmospheric effects during 
30 rendering of a scene such as a landscape, by increasing tiie degree of haziness 
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assodated with distant objects or by shifting the color of distant objects toward 
the blue with an increase in their apparent distance. Still other psychological 
vision cues which give the appearance of three dimensional depth to a scene are 
a reduction ui the brightness of objects percehred as bemg in the distance or a 

5 loss of focus of such objects. 

[ 0129] Yet another psychological vision cue is the use of occlusion, which 
means that portions of a more distant object may be obscured by objects in die 
foreground. Volumetric displays are not able to provide true physical ocdusion 
within the 3D images because foreground portions of the image cannot block 

10 ±e Ught from background portions of the image. Thus, if both the foreground 
and background portions of the 3D image are generated in their entirety, tiie 
background portion will be seen through the foreground portion, making the 
foreground portion appear translucent rather than solid. However, a quasi- 
occlusion effect can be created by not generating tiiose portions of background 

15 images that would otherwise be occluded by foreground images. Thus, at least 
within an angular range about a selected viewing axis, one can obtain an 
apparent ocdusion effect by this technique. 

[ 0130] Although use of psychological vision cues are well-known to 
painters and artists desiring to unpart a tiiree-dmiensional quality to two- 

20 dimensional paintings, etc., we have discovered that the combination of sudi 
psychological vision cues, when combined with die physical cues inherentiy 
provided by the volumetric three-dimensional displays disdosed herein, provide 
3D images whose apparent deptii can exceed the physical depth of the MOE 
device 32, sometimes by a large factor. 

25 [ 0131] For example, an image of tiie interior of a 3D box may be 
rendered into a 3D volumetric image by die system disdosed herein. By 
rendering tiie box in geometricaUy accurate fashion, die interior of the box 
would appear no deeper tiian tiie deptii of tiie display (i.e., die deptii of MOE 
device 32). However, by employing forced perspective during rendering of tiie 

30 3D box prior to forming tiie volumetric image, whereby tiie deeper parts of die 
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image are rendered at a reduced scale, the 3D box can be made to appear 
considerably deeper than it would otherwise appear in the three-dimensional 
image. 

[ 0132] By way of another example, an image of a road receding into the 
5 distance within a volmnetric display can be made to appear considerably more 
realistic through a combination of the physical depth of the display and the use 
of both forced perspective and a reduction of image resolution wi± distance, as 
could be implemented by low pass fQtering during the rendering process. 
[ 0133] As should be evident from the foregoing, it may be advantageous 
10 to add one or more of the aforementioned psychological visual cues, as well as 
others, during rendering of a scene prior to projection of the scene to form a 
volumetric 3D image. 

[ 0134] In implementing the MVD system, the psychological vision cues 
can be added during the rendering process within the MVD system 10 by using 

15 commercially available software applications such as 3D Studio Max, Softimage, 
and Lightwave. These software applications could be resident in graphics data 
source 16, MVD controller 18 or could be included in a separate stand-alone 
processor that is functionally part of the MVD controller 18. As an example, a 
background blur attributable to a short depth of focus is a psychological vision 

20 cue tiiat can be added by compositing togeflier a nimiber of renderings of a ' 
scene, each rendering being created with the camera pivoted slightiy around the 
point of focus. 

[ 0135] The psychological vision cues of haze, blue shifting of light with 
deptii, dinumng of brightness with deptfi, and deptii of focus (i.e., atmospheric 

25 psychological cues) can also be added in real tiihe by the input processor of the 
graphics data source 16, MVD controller 18, or a separate processor that is part 
of MVD controller 18. More specifically, image data transferred to the displays 
frame buffer may be stored in such a way that images at different depths are in 
separate storage areas. This enables depth dependent image processing to be 

30 carried out to introduce atmospheric cues. For example, haze can be added by 
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redudng the contrast of deeper images. Blue shifting can be added by shifting 
the color balance of deeper images toward the blue. Dimming can be added by 
reducing the brightness of deeper images. Depth of focus blur can be added by 
applying a Gaussian blur fUter of increasing strength to images of increasing 

5 distance on either side of the focus depth. 

[ 0136] Physical and/or psychological depth cues are often added to 
enhance the display of 2D images to give them a "3D" appearance, for example 
as set forth in U.S. Patent No. 5,886,818, with respect to enhancing 2D images 
which are projected so as to appear floating in space. However, it has 

10 previously not been recognized that physical and psychological depth cues, 
including but not limited to those described above, can also significantiy 
enhance the 3D appearance of the volumetric 3D images generated by the 
systems and techniques disclosed herein. Thus, notwithstanding the fact that a 
volumetric 3D image is generated by ±ese systems and techniques, the addition 

15 to that 3D image of physical and/or psychological depth cues during the image 
rendering process serves to create 3D volumetric images that are perceived as 
being even more realisticaUy in three dunensions than would otherwise be the 
case in die absence of such cues. 

[ 01373 In another embodiment, the sUces 24-30 may be generated and 
20 projected such that some of the images 44-50 are respectively displayed on 
more than one of the optical elements 36-42. in order to oversample the depth 
by displaying the images over a range of deptfis in the MOE device 32 instead of 
at a single depth corresponding to a single optical element. For example, 
oversampling may be advantageous if the MOE device 32 has more planes of 
25 optical elements 36-42 than the number of image sUces 24-30, and so the 

number of images 44-50 is greater than die number of image sUces 24-30. For 
example, a slice 24 displayed on both of optical elements 36-38 as images 44- 
46, respectively. Such oversampling generates the 3D image 34 with a more 
continuous appearance without increasing the number of optical elements 36- 
30 42 or the frame rate of the unage projector 20. Such oversampUng may be 
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perfonned, for example, by switching multiple optical elements to be in an 
opaque state to receive a single projected slice during a respective multiple 
projection cycles onto the respectively opaque multiple optical elements. 

ALTERNATIVE EMBODIMENTS OF THE MVD SYSTEM 
5 [ 0138] In one alternative embodiment, the MOE device 32 includes 10 
liquid crystal panels 36-42 and is dimensioned to be 5.5 inches (14 cm) long by 
5.25 inches (13.3 cm) wide by 2 inches (4.8 cm) in depth. The unage projector 
20 includes an acousto-optical laser beam scanner using a pair of ion lasers to 
produce red, green, and blue light, which was modulated and then scanned by 

10 high frequency sound waves. The laser scanner is capable of vector scanning 
166,000 points per second at a resolution of 200 x 200 points. When combined 
with the 10 plane MOE device 32 operating at 40 Hz, the MVD system 10 
produces 3D images with a total of 400,000 voxels, that is, 3D picture elements. 
A color depth of 24-bit RGB resolution is obtained, with an image update rate of 

15 1 Hz. Using a real image proj.ector 54, a field of view of 100* x 45' can be 
attained. 

[ 0139] In another alternative embodiment, the MOE device 32 todudes 
12 liquid crystal panels 36-42 and is dimensioned to be 6 inches (15.2 cm) long 
by 6 inches (15.2 cm) wide by 3 inches (7.7 cm) in depth. The image projector 

20 20 includes a pair of TEXAS INSTRUMENTS DLP video projectors, designed to 
operate in field sequential color mode to produce grayscale images at a frame 
rate of 180 Hz. By interlacing the two projectors, an effectively single projector 
is formed widi a frame rate of 360 Hz, to produce 12 plane volumetric images 
at a rate of 30 Hz. The transverse resolution attainable is 640 x 480 points. 

25 When combined with the 12 plane MOE device 32 operating at 30 Hz, die MVD 
system 10 produces gray 3D images with a total of 3,686,400 voxels. Using a 
real image projector 54, a field of view of 100° x 45' can be attained. 
[ 0140] In a further alternative embodiment, the MOE device 32 includes 
50 liquid crystal panels 36-42 and is dimensioned to be 15 inches (38.1 cm) 
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long by 13 inches (33.0 cm) wide by 10 inches (25.4 cm) in depth. The image 
projector 20 includes a high speed analog ferroelectric LCD available from 
BOULDER NONLINEAR SYSTEMS, which is extremely £ast vnA a frame rate of 
about lOkHz. The transverse resolution attainable is 512 x 512 points. When 
5 combined with die 50 plane MOE device 32 operating at 40 Hz, the MVD 
system 10 produces 3D images with a total of 13,107,200 voxels. A color depth 
of 24-bit RGB resolution is obtained, wi± an image update rate of 10 Hz. Using 
a real image projector 54, a field of view of 100* x 45* can be attained. With 
such resolutions and a volume rate of 40 Hz non-interfaced, the MVD system 10 
10 has a display capability equivalent to a conventional monitor wi± a 20 inch 
(50.8 cm) diagonal. 

[ 0141] In another embodiment, the optical elements 36-42 may have a 
transverse resolution of 1280 x 1024 and a depth resolution of 256 planes. The 
system will potentially operate in a depth interlaced mode in which alternated 

15 planes are written at a total rate of 75 Hz, widi the complete volume updated at 
a rate of 37.5 Hz. Such interlacing provides a higher perceived volume rate 
writhout having to increase the frame rate of the image projector 20. 
[ 0142] In a further embodunent, the MOE device 32 includes 500 planes 
for a significantly large depth resolution, and a transverse resolution of 2048 x 

20 •• 2048 pbcek, which results in a voxel count greater tiian 2 biUion voxels. The 
size of the MOE device 32 in this configuration is 33 inches (84 cm) long by 25 
inches (64 cm) wide by 25 inches (64 cm) in depth, which is equivalent to a 
conventional display witii a 41 inch (104 cm) diagonal. The image projector 20 
in tills embodiment includes ±e Grating Light Valve technology of SILICON 

25 LIGHT MACHINES, to provide a frame rate of 20 kHz. 

VIRTUAL INTERACTION APPLICATIONS 
[ 0143] Alternative embodiments of tiie MVD system 10 incorporating die 
user feedback device 58 as a force feedback interface allow the viewer 12 to 
perceive and experience touching and feeUng tiie 3D images 34, 56 at tixe same 
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location where the 3D images 34, 56 appear. The MVD system 10 can generate 
high resolution 3D images 34, 56 and so virtual interaction is implemented in 
the MVD system 10 using appropriate force feedback apparatus to generate high 
resolution surface textures and very hard surfaces, that is, surfaces which 
5 appear to resist and/or to have low compliance in view^ of the virtual reaUty 
movements of portions of the surfaces by ±e viewer 12. 
[ 0144] Accordingly, die user feedback device 58 includes high resolution 
position encoders and a high frequency feedback loop to match the movements 
of the hands of the viewer 12 with modifications to the 3D unages 34, 56 as 

10 well as force feedback sensation on the viewer 12. Preferably, the user feedback 
device 58 includes lightweight and compact virtual reality components, such as 
force-rfeedback-inducing gloves, in order that the reduced mass and bulk and the 
associated weight and inertia of the components impede the motions of the 
viewer 12 at a minimum. 

15 [ 0145] Such user feedback devices may include lightweight carbon 

composites to dramatically reduce the weight of any wearable components worn 
by the viewer 12. Furthermore, very compact and much higher resolution fiber 
optic or capacitive position encoders may be used instead of bulky optical 
position encoders know in the art to determine the position of portions of the 

20 viewer 12 such as hands and head orientations, 

[ 0146] The wearable component on the viewer 12 include embedded 
processor systems to control the user feedback device 58, thus relieving the 
processing overhead of the MVD controller 18 and/or interface 14. By using an 
embedded processor whose only task is to run the mterfece, the feedback rate 

25 for the overall MVD system 10 may be greater tiian 100 kHz. When combined 
witii very high resolution encoders, the MVD system has a dramatically high 
fidelity force feedback interface. 

[ 0147] Using such virtual interaction technologies with the MVD system 
10 which is capable of displaying such volmnetric 3D images 34, 56, a 3D GUI is 
30 implemented to allow a viewer 12 to access and directiy manipulate 3D data. 
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Known interface devices such as the data glove, video gesture recognition 
devices, and a FISH SENSOR system available from the MIT MEDIA LAB of 
Cambridge, Massachusetts, can be used to allow a user to direcdy manipulate 
3D data, for example, m 3D graphics and computer aided design (CAD) 
5 systems. 

[ 0148] For such 3D image and data manipulation, the MVD system 10 
may also incorporate a 3D mouse device, such as the SPACE BALL available 
from Spacetec Inc. of IxmeVi; Massachusetts, as well as a 3D pointing device 
which moves a 3D cursor anywhere in the display volume areas around image 

10 34 in die same manner as a viewer 12 moves one's hand in true space. 
Alternatively, the MVD system 10, through user feedback device 58, may 
interpret movement of the hand of the viewer 12 as the 3D cursor. 
[ 0149] In one embodiment, the user feedback device 58 may include 
components for sensing the position and orientation of the hand of the viewer 

15 12. For example, the viewer 12 may hold or wear a position sensor such as a 
magnetic sensor available fro POLYHEMUS, INC., and/or other types of sensors 
such as positional sensors incorporated in virtual reality data gloves. 
Alternatively, die position of the hand is sensed within the volume of the display 
of the 3D image 34 through the use of computer image processing, or a radio 

20 frequency sensor such as sensors developed at the MIT MEDIA LAB. To avoid 
muscle fatigue, die user feedback device 58 may sense the movement of a hand 
or a finger of the viewer 12 in much smaller sensing space tiiat is physically 
separate from the displayed 3D image 34, in a manner similar to 2D movement 
of a conventional 2D mouse on the flat surface of a desktop to control the 

25 position of a 2D cursor on a 2D screen of a personal computer. 

ADVANTAGES OF THE MVD SYSTEM 
[ 0150] Using the MVD system 10, the 3D images 34, 56 are generated to 
provide for natural viewing by the viewer 12, that is the 3D images 34, 56 have 
substantially all of the depth cues associated with viewing a real object, which 
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minimizes eye strain and allows viewing for extended periods of time without 
fatigue. 

[ 0151] The MVD system 10 provides a high resolution/voxel count with 
the MOE device 32 providing voxel counts greater than, for example, 3,000,000 
5 which is at least one order of magnitude over many volumetric displays knovsm 
in the art. In addition, by preferably using a rectilinear geometry for displaying 
the 3D image 34, such as a MOE deice 32 having a rectangular cross-section 
adapted to displaying image slices 24-30 as 2D images 44-50, the MVD system 
10 uses a coordinate system which matches internal coordinate systems of many 

10 known graphics computers and graphical applications programs, which 
facilitates and maximizes computer performance and display update rate 
without requiring additional conversion software. Additionally, in a preferred 
embodiment, the image voxels of the MOE 32 have identical and constant 
shapes, sizes, and orientations, which thus eUminates image distortion in the 3D 

15 image 34. 

[ 0152] Unlike multiview autostereoscopic displays known in the art, the 
MVD system 10 provides a wide field of view with both horizontal and vertical 
parallax, which allows the 3D image to be "looked around" by the view in 
multiple dimensions instead of only one. In addition, unlike multiview 
20 autostereoscopic displays, the field of view of die MVD system 10 is continuous 
in all directions, that is, there are no disconcerting jumps in the 3D image 34 as 
the viewer 12 moves with respect to the MOE device 32. 
[ 0153] Further, due to ±e static construction of the optical elements 36- 
42 in the MOE device 32, there are no moving parts which, upon a loss of 
balance of the entire MOE device 32, results in image distortions, display 
vibrations, and even catastrophic mechanical failure of the MOE device 32. 
[ 0154] The MVD system 10 may also avoid occlusion, that is, the 
obstruction by foreground objects of light emitted by background objects. A 
limited form of occlusion, called computational occlusion, can be produced by 
picking a particular point of view, and then simply not drawing surfaces that 
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cannot be seen from that point of view, in order to improve the rate of image 
construction and display. However, when the viewer 12 attempts to look 
around foreground objects, the parts of background objects that were not drawn 
are not visible. In one embodiment, the MVD system 10 compensates for the 
5 lack of occlusion by interspersing scattering optical element displaying an image 
with other optical elements in a scattering state to create occlusion by absorbing 
background light. Guest host polymer dispersed liquid crystals maybe used in 
the optical elements 36-42, in which a dye is mixed with the Uquid crystal 
molecules, allowing the color of the material to change with appUed voltage. 
10 [ 0155] The MVD system 10 also has Uttle to no contrast degradation due 
to ambient illumination of the MVD system 10, since the use of the real image 
projector 54 requires a housing extending to the MOE device 32, which in turn 
reduces the amount of ambient Ught reaching the MOE device 32, and thereby 
prevent contrast degradation. 
15 [ 0156] Alternatively, contrast degradation can be reduced by increasing 
the illumination from the image projector 20 in proportion to die ambient 
illumination, and by installing an absorbing plastic enclosure around the MOE 
device 32 to reduce the image brightness to viewable levels. The ambient Ught 
must pass through the absorbing enclosure twice to reach the viewer 12 - once 
20 on the way in and again scattering off the optical elements 36-42 of the MOE 
device 32. On the contrary, the Ught from the image projector 20 which forms 
the images 44-50 only passes tfirough tiie absorbing enclosure on the way to tiie 
viewer 12, and so had a reduced loss of iUumination, which is a function of the 
square root of the loss suffered by ambient Ught. 
25 [0157] An alternative embodunent reduces the effects of ambient Ught is 
to sue an enclosure with three narrow spectral bandpasses in the red, green and 
blue, and a high absorption for out-of-band Ught, which is highly effective to 
reduce such ambient Ught effects. Greater performance in view of ambient Ught 
is obtained by using laser Ught sources in die image projector 20, since tiie 
30 narrowband Ught from laser light sources passes unattenuated after scattering 
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from the MOE device 32, while the broadband light from the ambient 
illumination is mostly absorbed. 

ANTI-ALIASING IN THE MOE DEVICE 
[ 0158] In another alternative embodiment, referring to FIG. 16 and as 
5 described herein, prior to transmission of the image data to the image projector 
20 and thence to the optical elements 160-168 of the MOE device 32, the MVD 
controller 18 or alternatively the graphics data source 16 may perform 3D anti- 
aliasing on the image data to smooth the features to be displayed in the 3D 
image 34 on the optical elements 160-168. Using 3D anti-aliasing, tiie system 
10 10 avoids imaging jagged lines or incomplete regions in depth, for example, 
between parallel planes 162-164 along the z-direction, due to display 
pixelization caused by the inherendy discrete voxel construction of the MOE 
device 32 with the optical elements 16-168 aligned in x-y planes normal to a z- 
axis. 

15 [ 0159] As die data corresponding to the image slices is generated, an 
image element 170 may appear near an edge of a plane transition, that is, 
between optical elements, for example, die optical elements 162-164. For 
illustrative purposes only, the configuration of the optical elements 160-168 and 
the voxel 170 therein shown in FIGS. 16-18 is exaggerated to more clearly 

20 describe and illustrate the disclosed anti-aliasing system and method, and so it 
is to be understood ±at the optical elements 160-168 may have relatively small 
spadngs therebetween. 

[ 0160] To avoid an abrupt transition at the specific image element 170 
and in the 3D image illuminated on the optical elements 162-164 from the 
25 projector 20 may be generated such that each of the images 172-174 on tiie 
optical elements 162-164, respectively, includes the image element 170 or a 
portion or derivative form thereof, and so the image element 170 is shared 
between both planes formed by the optical elements 162-164, which softens the 
transition and allows tiie 3D image 34 in FIG. 1 to appear more continuous. 
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The brightness of the image elements 172-174 on respective consecutive optical 
elements 162-164 is varied in accordance with the location of the image 
elements 172-174 in the image data. 

[ 0161] Referring to FIG. 16, the number N of optical elements 160-168 
. 5 may be planar LCD surfaces, and so may be labeled Pj, Pj, P3, Pn. span a 
distance D being the width of the MOE device 32. Accordingly, each of the 
optical elements 160-168 maybe spaced at distances Dj, Dj, D3, ... along the 
z-axis from a common reference point, such that D^-D^ = D. For example, the 
common reference point may be the optical element 160 dosest along the z-axis 
10 to the projector 20, so 0^=0 and D„=D. Alternatively, the distances of the 

optical elements 160-168 may be measured from the lens 22 of the projector 20, 
so an offeet distance Dopps^from the optical element 160 and the lens 22 may 
be subtracted from absolute distances D^, D^, D3, ... of the optical elements 
• 160-168 from the lens 22 to obtain relative distances from the optical element 
15 160. Accordingly, D,=Do^. The optical elements 160-168 may also have a 
uniform spacing S therebetween, or alternatively the spacing between the 
optical elements 160-168 may vary. 

[ 0162] As described herein, a depth value of each voxel 170 is measured 
along the z-axis from a reference point either at the lens 22 or at the optical 
20 element 160, and such depth values are stored in a depth buffer with an 

associated color value stored in a color buffer. For example, a depth value Dv is 
associated with the voxel 170. 

[ 0163] To perform anti-aliasing and tiius to smooth the appearance of the 
voxel 170 lying between die optical elements 162-164, the distances D^, Db 

25 between ±e depth value Dvand die optical elements 162-164, respectively, are 
determined, and such distances are used to generate an anti-aUasing parameter. 
The anti-aUasing parameter to generate respective color values for die two 
voxels 172-174 on tiie optical elements 162-164, respectively witii tiie 
corresponding color value of die voxel 170 being modified by the anti-aliasing 

30 parameter to generate respective color values for the two voxels 172-174. 
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[ 0164] FIG. 17 illustxates a voxel display without the use of anti-aliasing. 
As shown in FIG. 17, the voxels 176-178 on the optical element 162 and the 
voxels 180-184 on the optical dement 164 fonn a sharp transition at the 
boundary defined by the voxels 178-180. If the distance between the optical 

5 elements 162-164 is significant, a noticeable jagged or broken appearance of 
image 34 may be formed by the combination of displayed voxels 176-184. For 
example, the voxels 178-180 may have had depth values between the optical 
elements 162-164, for example, with the voxel 178 being closer to but not on 
the optical element 162 and the voxel 180 being closer to but not on die optical 

10 element 162. Such intermediate depth values may then have been converted to 
the discrete depth values of the optical elements 162-164, respectively, in 
order to display the voxels 178-180. Further, the color values of the voxels 178- 
180 in FIG. 17 are unchanged, and so the intensity of the color of die voxel 178- 
180 may appear anomalous for such differing optical depths. In die alternative, 

15 the voxels 178-180 at the transition may be omitted due to their intermediate 
depths, but then the 3D image 34 composed of voxels 176 and 182-184 may 
appear to have holes or firactures. 

[ 0165] Using anti-aUasing, as shown in FIG. 18, both transitional voxels 
178-180 may be used to generated new voxels 178A-178B and 180A-180B, with 

20 the voxels 178A-178B displayed on the optical element 162 and the voxels 
178B-180B displayed on the optical element 164. In addition, as shown in FIG. 
18, whUe die color values of the new voxels may be modified such unchanged, 
by performing anti-aUasing, ±e color values of die new voxels may be modified 
such that each of tiie new voxels 178A-178B and 180A-180B has an adjusted 

25 color to soften the image transition in die x-y plane across different depths. 
Accordingly, as shown in FIG. 19, while tiie voxels 176-184 have an abrupt 
transition in apparent deptii according to the curve 176 for the imaging in FIG. 
17, die voxels 176, 178A-178B, 180A-180B, and 182-184 in FIG. 18 have a 
relatively smoother transition in apparent depdi according to the curve 188. It 

30 is noted that, for iUustrative purposes only, die curves 186-188 are not overlaid 
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in FIG. 18 in order to dearly show the curves 186-188, and so it is to be 
understood that, in FIG. 18, the apparent depths of voxels 176 and 182-184 are 
identical widi and vyithout anti-aliasing. 

[ 0166] In FIG. 19, the voxels 178A-178B of FIG. 18 form an image aaoss 
5 the optical elements 162-164 with an apparent depth 178C intermediate 
between the depths of the voxels 178A-178B and corresponding to the original 
depth of the voxel 178 in FIG. 17 to be closer but not on the optical element 
162. Similarly, the voxels 180A-180B of FIG. 18 form an image across the 
optical elements 162-164 with an apparent depth of 180C intermediate between 
10 the depths of the voxels 180A-180B and corresponding to the original depth of 
the voxel 180 in FIG. 17 to be closer but not on the optical element 164. 
[ 0167] It is to be understood that the anti-aliasing is not limited to the 
nearest two bounding optical elements, but instead the voxels 178-180 may be 
used to generate a plurality of corresponding voxels on a respective plurality of 
15 the optical elements 160-168. and so to provide depth transition curves which 
may be, for example, smoother than the curve 188 m FIG. 19. For example, the 
depdi transition curve 188 due to anti-aliasing may approxunate a sigmoid or 
tangent function. 

[ 0168] Referring to FIG. 16, to perform anti-aliasing for the voxel 170, at 
20 least one depth adjustment value 1 is generated which is a function of the 
distance of the voxel 170 from at least one optical element. In one 
embodiment, adjustment values 1, m may be generated which are functions of 
scaled values of the distance D^, Db from die respective optical elements 162- 
164. The adjustment values 1. m are tiien used to modify a color value O, 
25 associated with the voxel 1 70 to generate new color values C^, Cg associated 
with the newly generated voxels 172-174, respectively, with the voxels 172-174 
having respective x-y positions on die optical elements 162-164 identical to die 
x-y position of the voxel 170. 

[ 0169] The color value of a voxel may specify at least the brighmess of 
30 die voxel to be displayed. Alternatively, tiie voxel 170 may be associated widi a 
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set of parameters including at least one scalar specifying the brightness of the 
colorized voxel. Accordingly, modification of the color values may be 
performed through multiplication of the color value by an adjustment value. 
For example, for a color value Q = 12 brightness units and an adjustment value 
5 \= .5, the modify color value is determmed to be Q, A,= (12 brightness 
units) X (.5) = 6 brightness units. 

[ 0170] In one embodiment, the distance Dy is scaled to be a depth value 
from 1 to N, in which N is the number of optical elements 160-168 and each of 
the integer values 1 to N corresponds to a specific one of the optical elements 
10 160-168, for example, as indices for the label Pj, P2, P3, & Pn shown in FIG. 16. 
The adjustment values 1, m are determined from the scaled depth value. If the 
optical elements 160-168 are uniformly spaced with constant spacing S along 
distance D, then: 

15 so a scaled distance of the voxel 170 is: 

Dy - DOFFSBT 
DSCAI£D= +1 C2) 

in which Dy is the absolute distance measured from the lens 22 or other 
reference points. For example, with the lens 22 being the origin of the z-axis, 
the optical element 160 may be at distance Dj = Doffset- 

20 [ 0171] DscALED is a real numbered value such that 1 i Djcaled ^ N, so the 
fractional portion of Dscai^d, which ranges between 0 and 1, indicated the 
relative distance from the optical elements 162-164. For the optical elements 
162-164 bounding the voxel 170 on either side along the z-axis, the indices of 
the optical elements 162-164 are: 

25 LDscaudJ and O) 

LDscaledJ +1. ("^^ 
respectively, in which LXj is the floor or integer ftinction of a value or variable X; 
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that is a function returning the largest integer less than X. 
[ 0172] The fractional portion of Dsc^ied is: 

and thus: 

5 li = l-A (6) 

The color values C^, Cb indicating respective brightnesses associated witfi 
the voxels 172, 174, respectively, are assigned tiie values: 

Ca := Cv Cl-^) C7) 
Cb:=Cv>' = CvC1-Ji) (8) 
10 in which tiie symbol ": =" indicated assignment of a new value. 

[ 0173] For example, for a voxel 170 having a depth Dv = 9.2 units from 
die lens 22, witii an offset Doppset = 3-0 units, with die MOE device 32 having 
five evenly-spaced optical elements extending twenty units in length, N=5, 
D=20, then the spacing S=5 umts, as per Equation (1), and Dscaled=2-24, 
15 accordingly to Equation (2). The voxel 170 is dius positioned between die 
optical having indices = 2 and [Dscai^J + 1=3, as per Equations 

(3).(4), and so in FIG. 16, the optical elements 162-164 having labels and P3 
are identified as the optical elements upon which new voxels 172-174 are to be 
displayed corresponding to the voxel 170. 
20 [ 0174] In this example, from Equations (5)-(6), die fractional value of die 
scaled depdiis X=.24 and so n=.76. Accordingly. (1-A)=.76 and (l-n)=.24 
and from Equations (7)-(8), the color value of die voxel 172 is Ca=.76Cv=76% 
of the brightness of die original voxel 170, and die color value of the voxel 174 
is Cb=.24Cv=24% of the brighmess of die original voxel 170. Thus, since the 
25 voxel 170 is "doser" to die optical element 162 dian die optical element 164, 
the corresponding new voxels 172-174 have a distributed brightness such tiiat 
die closer optical element 162 displays die majority of die color between die 
two voxels 172-174, while die fardier optical element 164 contributes a lesser 
but non-zero amount to die appearance at die transition of die 3D volumetric 
30 image between die optical elements 162-164 at die voxel 170. 
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[ 0175] For the voxels 170 have depth values lying precisely on optical 
elements 160-168, no anti-aUasing is required. Accordingly. Equations (2)-(4) 
degenerate to integer values, and Equations (5)-(6) result in the adjustment 
values k ,n being 0 and 1, respectively, or bemg 1 and 0. respectively, so no 

5 adjustment of the color values is performed. To avoid unnecessary 

computation, the MVD controUer 18 may check whether the computation in 
Equation (2) results in an integer, within a predetermined error tolerance such 
as 1 percent, and if so, the voxel 170 is determined or deemed to lie precisely 
on one of the optical elements 160-168. The anti-aliasing procedure is 

10 terminated for the currentiy processed voxel 170, and the procedure may then 
continue to process other voxels of 3D image 34. 
[ 01761 In this embodiment using Equations (l)-(B), since uniform 
spacing and odier characteristics of the MOE device 32 are known, no search for 
the nearest bounding optical elements is necessary, since the distance Dv of the 

15 voxel 170 and the MOE device characteristics determine which optical elements 
bound the voxel 170, by Equations (3)-(4). 

[ 0177] In another alternative embodiment, for optical elements 160-168 
of an MOE device 32 having either uniform spacing, or having variable and/or 
non-uniform spacing, the anti-aUasing may be performed using Equations (9)- 

20 (13) set forth below in conjunction with Equations (7)-(8) above. For example, 
for MOE devices having variable spacing and/or variable offsets of the MOE 
device from the projector 20 and lens 22, the anti-aliasmg mediod may be 
performed on-tiie-fly during modification of the spacing and configuration of 
the optical elements 160-168. Since die distances/depths of die optical elements 

25 160-168 may vary, in tiie alternative embodiment, the anti-aliasing method 
determines at least two optical elements bounding the voxel 170 currently being 
processed, by searching the deptfi values of each of die optical elements 
160-168 for the two bounding optical elements having a distance/deptii values 
Dneari ancl DNE^iuSUch tiiat: 

30 DneaIU^^V^^NEAR2 
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[ 0178] The variables NEARl and NEAR2 may be integer indices 
specifying the associated optical elements from among the optical elements 160- 
168. For example, in FIG. 16, NEARl = 2 and NEAR2 = 3, corresponding to 
the optical elements 162-164 boimding the voxel 170 along the z-axis. 
[ 0179] The depth adjustment values A, are determined to be: 

Dv - Dneari 



Dneari - Dneari 



(10) 



Dv- 


Dneari 


Dnearx 


— Dneari 



(11) 



10 



in which |X| is the absolute value or magnitude function of a value or variable 
X. 

[ 0180] The depth adjustment values from Equations (lO)-(ll) are botii 
positive real numbers which satisfy: 

(12) 

X + ^ = 1 (13) 
and so the depth adjustment values scale the non-uniform and/or variable 
15 distances between optical elements, and are ±en used in Equations (7)-(8) to 
generate the voxels 172-174 with the corresponding adjusted color values. As 
shown in Equations (10) -(11), the depth adjustment values 1, m are based on 
interpolations of the depth of die voxel 170 witiiin the range of deptfis of the 
voxels 172-174 associated with the optical elements 162-164, respectively. 
[ 0181] In the above example having uniform spadng. Equations (9)- (13) 
are applied to with Dy = 9.2 units, Dneari = Da = 8 units and D^q^ = D3 = 13 
units, so: 



20 



9.2 


-8 


8- 


13 



92-13 



8-13 



= ^=24 
5 

3-8 , 
= y=.76 



25 which agrees with the adjustment values using Equations (l)-(8). The 
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alternative embodiment is useful if the dimensional and spatial characteristics of 
the MOE device 32 and the optical elements 160-168 vary, but a search is 
required to determine the appropriate bounding optical elements 162-164 for 
generating the nev\^ voxels 172-174. 
5 [ 0182] FIG. 20 illustrates a flowchart of a method implementing 3D anti- 
aliasing as described herein, in which, for a current voxel to be displayed, such 
as the voxel 170, the method reads the corresponding depth value Dy and the 
color value Cv from the depth and color buffers, respectively, in step 190. The 
method may then determine if the spacing between the optical elements 
10 constant in step 192; for example, a configuration setting of the MVD controUer 
18 may indicate if the optical elements 160-168 are fixed, having uniform or 
non-uniform distribution, and/or die MVD controller 18 and the MOE device 32 
operate in a variable spacing mode, as describe herein. 
[ 0183] If the spacing is constant, die method tfien scales the depth value 
15 Dv in step 194 to be v^thin the range of indices of the optical elements 160-168 
using Equations (l)-(2) , and tiien the mediod determines die optical elements 
nearest to an bounding the depth value Dyin step 196 using Equations (3)-(4) 
in step 196. Otherwise, if the sparing is not constant in step 192, the method 
may perform step 196 without step 194 ui the alternative embodiment to 
20 determme the optical elements satisfying Equation (9); that is, using a search 
procedure durough the distance/depth values of each of the optical elements 
160-168. In another alternative method, the step 192 may be optionally 
implemented or omitted, depending on die configuration and operating mode of 
the MVD controller 18 and the MOE device 32. 
25 [ 0184] The mediod then determines a deptii adjustment value X and/or a 
second value \i in step 198 using Equations (5)-(6) or Equations (lO)-(ll), 
depending on die embodiment implemented as described herein. The mediod 
tiien adjusts the color values in step 200 for voxels on die nearest bounding 
optical elements using die deptii adjustment value or values using Equations 
30 (7)-(8) and die mediod displays die adjusted voxels in step 202 on die nearest 
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bounding optical elements with the adjusted color values. 
[ 0185] In another alternative embodiment, an intermediate degree of 
anti-aliasing may be implemented. For example, the adjustment values X, \i 
may be fixed to the value of, for example, .5, such that half of the brightness of 
5 the voxel 170 is assigned to each of the voxels 172-174. Such intermediate anti- 
aliasing may generate apparent depths such as an intermediate dep± 180D 
corresponding to intermediate transition curves such as shown by the curve 189 
in FIG. 19. 

[ 0186] In other alternative embodiments, the degree of anti-aliasing can 
10 be varied from one extreme; that is ignoring the fractional depth values X, \i to 
assign the color values; to another extreme of using all of the fractional depth 
values X, |i, or the degree of anti-aliasing can be varied to any value between 
such extremes. Such variable anti-aliasing may be performed by dividing the 
fractional portion 1 of the scaled depth by an anti-aliasing parameter P, and then 
15 negatively offsetting the resulting value from one. That is, after a is calculated 
in Equation (5) and (10), a variable Avar ^ calculated such that: 



[ 0187] The final color value may be determined by fixing or clamping the 
negatively offset value to be within a predetermined range, such as between 0 
20 and 1. Accordingly, Equations (7)-(8) are modified for variable anti-aliasing 
such that: 



[ 0188] The steps 198-202 in FIG. 20 may thus implement Equations (14)- 

25 (16), respectively, to provide variable anti-aliasing. 

[ 0189] An anti-aliasing parameter P= 1 corresponds to full anti-aliasing, 
and an anti-aliasing parameter of infinity, P*^«>, which may be implemented 
computationally with an arbitrary high nmnerical value, corresponds to no anti- 
ahasing. Anti-aliasing parameters less than 1 may also be implemented. For 

30 example, when P=l, anti-aliasing as described above for Equations (1)-(13) is 



^AR — ^ / P 



(14) 




(15) 
(16) 
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implemented. 

[ 0190] In another example, for an anti-aliasing value of X = .24 an anti- 
aliasing parameter of 3, =.08 by Equation (14) and so = .92Cy = 92% 
of the color value of the voxel 170, while Cga = .080^ = 8% of the color value of 
5 the voxel 170, as per Equations (15)-(16). Compared to the previous numerical 
example, such variable anti-aliasing increases the contribution of the voxel 172 
in the apparent depth from 76% to 92% while the voxel 174 has a decreased 
contribution, from 24% or about one-foiirth decreased to less than 10%. In a 
further example, P-<», anti-aliasing is eliminated, and so = 0.00 by 

10 Equation (14). Thus, C^a = (l.O)Cv =100 % of the color value of the voxel 170, 
while = (O.O)Cv = 0 % of the color value of the voxel 170, as per Equations 
(15)"(16). Accordingly, any voxels 170 lying between the optical elements 162- 
124 are displayed on the closer optical element 162, without anti-aliasing, and 
so step 202 in FIG.20 may further include the step of not generating and thus 

15 not displaying a second voxel farther from the reference point if P-«>. For 
example, the voxel 174 is no generated. 

[ 0191] In further alternative embodiments using variable anti-aliasing, 
the method in FIG. 20 may include displaying new voxels only if the adjusted 
color values are greater than a predetermined threshold T. For example, 
20 ifCv(l-^AR) >T then Ca2=Cv(1- Avar) (17) 

else = 0 

if CvA VAR > T then Cb2 = Cy Avar (18) 

else = 0. 

[ 0192] For example, T may equal .05, and so contributions of color less 
25 than 5 % may be considered negligible, for example, since voxels with such 
color values are displayed on the optical elements 160- 168 when switched to 
opaque/ scattering mode. Accordingly, such negligible contributions to the 
overall 3D image are discarded, and the non-contributing voxels are not 
displayed and improve computational processing of the 3D image. 
30 [ 0193] In additional alternative embodiments, the MVD system 10 is 
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capable of generating the 3D image 34 having the appearance of translucently 
of portions of the 3D image 34. That is. the images 44-50 displayed on the 
optical elements 36-42 of the MOE device 32 have appropriate shading and 
colors such that a portion of one image may appear tiranslucent. with anotiier 
S portion of a second image appearing to be viewable tiirough the translucent 
portion. Such translucent appearances may be generated with or without anti- 
aliasing. 

[ 0194] In generating the 3D image 34, the method employed by die MVD 
system 10 performs the PRD computation using, for example, OpenGL frame 
10 buffer data, such as the color and dep± (or z) buffers of the frame buffer of the 
graphics data source 16. A value in the depth buffer is tiie deptii of the 
corresponding pixel in tiie color buffer, and is used to determine the location of 
the pbcel or voxel, such as 170 in HG. 16, displayed within the MOE device 32. 
This MPD computation method is appropriate in situations in which it is desired 
15 that portions of the images of background objects of the volumetric image 34 
from the MOE device 323 are not rendered if such images are occluded by 
images of foreground objects. For generated images in the MOE device 32 in 
which the images of foreground objects are translucent to allow the image 
corresponding to an occluded background object to be seen an alpha channel 
20 technique is used, in which a parameter a (alpha) determines tiie color of a 
pixel/voxel in tfie color buffer by combining tiie colors of botii die foreground 
and background objects, dependhig on devalue of a. Total opacity is given by 
a = 1, and total transparency is given by a = 0. WhUe using such alpha channel 
imaging to generate color images from die color buffer diat look correct, die 
25 depdi values in ±e deptii buffer may he unchanged, and so stiU correspond to 
the depths of die images of die foremost objects. In known display systems, die 
unmodified depdis prohibit die proper display of images in die volumetiic 
display system since diere may be multiple surfaces at a variety of depdis which 
are to be displayed using only a single depdi value. The disclosed MVD system 
30 10 generates volumetiic images 34 having, for example, translucent objects or 
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portions thereof which avoids the prohibition in the prior art in displaying 
multiple surfaces at a variety of depths for a single depth value. The disclosed 
MVD system 10 uses additional features of OpenGL to generate cUp planes 
located in the model space of the MVD system 10. with which rendering is only 
5 allowed to occur, for example, on a predetermined side of each clip plane, such 
as a positive side as opposed to a negative side. 

[ 0195] For an MOE device 32 having N planes 204-212 which may be 
numbered v^th indices I to N and having a uniform spadng A therebetween, as 
shown in FIGS. 21-24. a scene such as a volumetric image 34 is rendered N 

10 times with the dip planes facing toward each other, separated by the distance A 
and centered on the location of a given MOE plane of the planes 204-212 in the 
model space. Thus, N different unages are generated, and the corresponding 
color buffer is retrieved from the frame buffer to be sent to the MVD controUer 
18. Upon sending the color buffer to die MVD controUer 18 for display in the 

15 MOE device 32, the alpha channel may be turned off since the MVD system 10 
has an inherent alpha value associated with the MOE device which is being 
used to generate the 3D volumetric unage 34. 

[ 0196] Rendering with dip planes may be implemented without anti- 
ahasing as shown in FIGS. 21-22, in whidi dip planes 214-216 are used 

20 corresponduig to image portions positioned doser to an observer 218, and 
portions of the image 34 are generated and displayed on a first plane 206 
positioned between the dip planes 214-216, with the image portions between 
die clip planes 214-216, displayed on the first plane 206. New portions of tl^e 
image 34 are generated between the dip planes 220-222 for display on a second 

25 plane 208 fartiier from die observer 218 and positioned between die cUp planes 
220-222, witii the image portions between the dip planes 220-222 displayed on 
the second plane 208. 

[ 0197] To implement anti-aUasing witii the above metiiod using t^e alpha 
diannel, other features of OpenGL are used, such as an atmospheric effect 
30 implementing fbg-like imaging used for the anti-aUasing. The fog feature causes 
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the color of each imaged object to he combined with the color of the fog in a 
ratio determined by the density of the fog and the depth of the model with 
respect to the depth range associated with far and near values specified for the 
fog. 

5 [0198] Fog ftmctionsavaUable in OpenGL include Unear, exponential, 
and exponential-squared functions. The disclosed MVD system 10 may use such 
functions, as well as combinations of such fog functions, such as the 
superposition's of linear fog functions 224-227 as shown in FIGS. 23-24. In an 
illustrative embodiment shown in FIGS. 23-24, each of the combinations of 
10 linear fog functions 224-227 starts with a value of zero, correspondtag to a 
black setting, at tiie near depth of tiie fog, and progresses in a Unear manner to 
a value of one, corresponding to a true-colors setting, at the distance (FAR- 
NEAR)/2 from the near depth location. The fog function then falls back to zero 
at die far depth of the fog. Witii such a fog function, and with tiie clip planes 
15 separated by a dismnce of 2A witii tiieir center being positioned on a given MOE 
plane in die model space upon which die image 34 is to be displayed, die image 
34 is rendered N times, and each time die data from die color buffer is sent to 
die corresponding plane of die MOE device 32. 

[ 0199] In an illustrative embodiment, die combination of linear fog 
20 functions and die processing of voxel image data widi such combinations are 
performed by syndiesizing images for a given optical element, such as die plane 
206 in FIG. 23. widi at least two rendering passes. During a first pass, two cUp 
planes are separated by die distance A widi a first cUp plane 228 positioned on 
an optical element 204 having images rendered diereon before die current 
25 optical element 206, and widi die second clip plane positioned on die current 
optical element 206. The forward Unear fog function 224. having distances 
inaeasing, widi NEAR less dian FAR. is dien used with die aforesaid clip planes 
to render a furst set of images for die optical element 206. 
[ 0200] During a second pass, die two dip planes are separated by die 
30 distance D, widi a first dip plane positioned on die current optical element 206, 
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and with the second dip plane 230 positioned on the optical element 208 to 
have images thereon rendered after the current optical element 206, and with 
the second clip plane portioned on the current optical element 206. The 
backward linear fog function 225, having distances increasing, with FAR less 
5 than NEAR, is then used vnth the aforesaid dip planes to render a second set of 
images for the optical element 206. 

[ 0201] The two sets of images rendered with the different Imear fog 
functions 224-225 are then added together by the MVD system 10 to be 
displayed on the optical element 206. 
10 [ 0202] For rendering a first image on a first plane 206 as shown in FIG. 

23, the fog functions 224-225 are centered about the first plane 206, and ±e 
images from the clip planes 228-230 and depths therebetween have their 
corresponding color values modified by tiie corresponding value of the fog 
functions 224-225 at the assodated depths. After rendering tiie added images 

15 on the optical element 206 using die functions 224-225, the MVD system 10 
proceeds to render a successive image on a second plane 208 as shown in FIG. 

24, widi the fog functions 226-227 being translated to be centered about die 
second plane 208. The images from die dip planes 232-234 and deptiis 
tiierebetween have tiieir corresponding color values modified by the 

20 corresponding value of the fog function 226 at the assodated deptiis. The MVD 
system 10 proceeds to successivdy move die fog function and to process 
corresponding clip planes for color adjustment of eadi respective image using 
the alpha channel method, hi alternative embodunents, different fog hmction 
may be implemented for different planes 204-212. for example, to have higher 

25 fog densities at greater distances from tiie observer 21 8 to increase deptii 
perceptive effects of tiie displayed 3D volumetric image 34. 
[ 0203] For example, referring to FIG. 23, for tiie images 236 at a deptii 
238 labeled D and having respective color values Cj, for each portion of tiie 
image, tiie value 240 of tiie fog frinction 224 at tiie deptii a^, so tiie adjusted 

30 color value displayed for tiie images 236 is a^C,. The color values C, may be tiie 
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depth adjusted color values as in Equations (7)-(8) and/or a5)-(18) as 
described herein, and so the alpha channel adjustments may be optionally 
implemented in step 200 of FIG. 20 to perform the anti-aliasing with the alpha 
channel techniques described herein. 
5 [ 0204] By the foregoing a novel and unobvious multi-planar volumetric 
display system 10 and method of operation has been disclosed by way of the 
preferred embodiment. However, numerous modifications and substitutions 
may be had witfiout departing from the spirit of tiie invention. For example, 
while the preferred embodiment discusses using planar optical elements such as 
10 flat panel liquid crystal displays, it is whoUy witiiin tiie preview of tiie invention 
to contemplate curved optical elements in the manner as set forth above. 
[ 0205] The MVD system 10 may be implemented using the apparatus and 
methods described in co-pending U.S. Provisional Patent Apphi. 
No. 60/082,442, filed April 20, 1998, as well as using the apparatus and 
15 mediods described in U.S. Patent No. 5,990,990 filed November 4, 1996. which 
is a continuation-in-part of U.S. Patent No. 5.572.375; which is a division of 
U.S. Patent No. 5,090,789. The MVD system 10 may also he implemented using 
tiie apparatus and methods described in co-pending U.S. Patent Appln. 
No. 09/004,722, filed January 8, 1998. Each of die above provisional and non- 
20 provisional patent applications and issued patents, respectively, are 

mcorporated herein by reference. Accordingly, die invention has been described 
by way of illustration rather than limitation. 



1733S7.1 



wo 03/017246 



PCT/US02/26718 



-62- 

WHAT IS CLAIMED IS: 

1. A system for displaying a volumetric three dimensional image of a 
three-dimensional scene having a virtual depth Dy, comprising: 

(a) a controller for generating data representing a series of two 
5 dimensional slices at varying depths through said three dimensional scene; 

Cb) a multi-surface optical device having a physical depth Dp 
including a plurality of optical elements; and 

(c) an image projector responsive to said data for selectively 
projecting said series of two dimensional sUces on respective optical elements 

10 of said multi-surface optical device to display a first volumetric three 

dimensional image viewable in the multi-surface optical device, wherein the 
virtual depth Dy of said tiiree dimensional scene is mapped to correspond to 
die physical depth of said multi-surface optical device; and 

(d) a projector for projecting tiie first volumetric tiiree 

15 dimensional image formed in tiie multi-surface optical device to generate a 
second volumetric ±ree dimensional image viewable in a space outside of said 
multi-surface optical device. 

2. The system of claim 1 wherein the virtual depth Dy of said tiiree 
dimensional scene is mapped to die physical deptii of said multi-surface 

20 optical device by a linear relationship. 

3. The system of daim 1 wherein die virtual depth Dy of said tiiree 
dimensional scene is mapped to tiie physical deptii of said multi-surface 
optical device by a non-linear relationship. 

4. The system of claim 1 wherein said plurality of optical elements 
25 are eqxially spaced along a depth axis. 
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5. The system of claim 1 wherein said plurality of optical elements 
are imequally spaced along a depth axis. 

6. The system of claims 4 and 5 wherein a depA within said first 
three dimensional image is further mapped to correspond to one of said 

5 pliurality of optical elements. 

7. A system for displaying a volumetric tiiree dimensional image of 
a three dimensional scene, comprising: 

(a) a controller for generating data representing a series of 
two dimensional slices at varying depths through a selected portion of a diree 

10 dimensional scene, said selected portion corresponding to a maximum virtual 
depth value Z^^x minimum virtual depth value Zmin; 

(b) a multi-surface optical device including a plurality of 
optical elements arranged; and 

(c) an image projector responsive to said data for projecting 
15 said series of two dimensional sUces onto respective optical elements of said 

multi-surface optical device to display a first volumetric tiiree dimensional 
image viewable m die multi-surface optical device, wherein die sUce associated 
witii said minimum virttial deptii value Zmd, is projected onto a forward optical 
element of said multi-surface optical device, and die sUce associated witii said 
20 maximum deptii value Zmax is projected onto a rearward optical element of said 
multi-surface optical device, widi remaining sUces being projected onto 
respective optical elements therebetween. 

8. The system of claim 7 wherein portions of said tiiree dunensional 
scene which lie beyond tiie maximum virtual deptii value Zmax are displayed as 
25 two-dimensional images on a rearward optical element 
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9. The system of claim 7 wherein the values of Z^^x ^mi 
assigned interactively by a viewer. 

10. The system of claim 7 wherein the values of Z^^x ^ 
assigned by a viewer during application startup. 

5 11. The system of daims 8 and 9 wherein the slice associated with 

said minimum depth value Zm,n is projected onto ±e forwardmost optical 
element of said multi-surface optical device and the sUce associated with said 
maximum depth value Z^^ is projected onto the rearmost optical element of 
said mvilti-siurface optical device. 

12. The system of claims 7 and 8 further including a floating image 
generator for projecting the first volumetric three dimensional image formed in 
the multi-surfece optical device to generate a second volumetric three 
dunensional image viewable as floating hi space at a location outside of the 
multi-surface optical device. 

15 13. A method for generating a volumetric tiiree dimensional image of 

a three dimensional scene, the mefliod comprising the steps of: 

(a) generating data representing a series of two dimensional 
sUces at varying depths tiu:ough a selected portion of said three dimensional 
scene, said selected portion corresponding a maximum depth value Zm^x and 

20 minimimi depth value Z^^; and 

(b) projecting the series of two dimensional sUces on 
respective optical elements of a multi-surface optical device to display a first 
volumetric three dimensional image viewable in said multi-surface optical 
device, wherein the sUce corresponding to said minimum depth value Zm,n is 
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projected onto a forward optical element of said multi-surfece optical device, 
and the sUce generated at said maximum depth value Z^^x is projected onto a 
rearward optical element of said multi-surfece optical device, with remaining 
slices being projected onto respective optical elements therebetween. 

5 14. Themethodof claim 13 wherein portions of said three 

dimensional scene which lie beyond tfie maximum depA value Zm^x are 
projected as two-dimensional images onto a rearward optical element. 

* 

15. The method of claim 13 wherein the values of Z^ax and Zmin are 
assigned interactively by a viewer. 

10 1 6. The method of claim 13 wherein the values of Z^^x and Z^in are 

assigned by a viewer during application startup. 

17. The method of claims 13 and 14 wherein the shce corresponding 
to said minimum depth value Z^in is projected onto the forwardmost optical 
element of said multi-surface optical device and the slice corresponding to said 

15 maximum depth value is projected onto the rearmost optical element of 
said multi-surface optical device. 

18. The method of clauns 13 and 14 further including the step of 
projecting die first volumetric three dimensional image from the multi-surface 
optical device to generate a second volumetric three dimensional image 

20 viewable as floating in space'at a location outside of die multi-surface optical 
device. 

19. A system for generating volumetric three dimensional images 
comprising: 

(a) a controller for rendering a three dimensional scene to 
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provide at least one psychological vision cue and for generating a series of two 
dimensional slices through said rendered three dimensional scene; 

Cb) an optical device onto which said two dimensional slices 
can be projected; and 

5 (c) an image projector for selectively projecting the series of 

two dimensional slices on said optical device to generate a first volumetric 
three dimensional image wherein said at least one psychological vision cue 
causes the apparent depth of said first volumetric three dimensional image to 
exceed the physical depth of said optical device. 

10 20 . The system of claim 19 wherein said at least one psychological 

cue is provided by including shading in said rendered three dimensional scene. 

21 . The system of claim 19 wherein said at least one psychological 
cue is provided by including shadows in said rendered three dimensional scene. 

22. The system of claim 19 wherein said at least one psychological 
15 cue is provided by occluding portions of said rendered three dimensional scene. 

23. The system of daim 19 wherein said at least one psychological 
cue is provided by including forced perspective in said rendered three 
dimensional scene. 

24. The system of claim 19 wherein said at least one psychological 
20 cue is provided by decreasing resolution in surface textures of more distant 

portions of said rendered three dimensional scene. 

25. The system of claim 19 wherein said at least one psychological 
cue is provided by increasing haziness of more distant portions of said rendered 
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three dimensional scene. 

26. The system of claim 19 wherein said at least one psychological 
cue is a provided by shifting the color of more distant portions of said rendered 
three dimensional scene towards blue. 

5 27. The system of claim 19 wherein said at least one psychological 

cue is added by reducing the brightness of more distant portions of said 
rendered three dimensional scene. 

28. The system of claim 19 wherein said at least one psychological 
cue is provided by reducing the focus of more distant portions of said rendered 

10 three dimensional scene. 

29. The system of Claim 19 wherein said optical device includes a 
plurality of optical elements. 

30. The system of Claim 29 wherein said plurality of optical elements 
are arranged along a depth axis. 

15 31. The system of claim 19 further including a floating image 

generator for projecting ±e first volumetric three dimensional image from the 
optical device to generate a second volumetric three dimensional image 
viewable as floating in space at a location outside of the optical device. 

32. A method for generating a volumetric three dimensional images 
20 of a three dimensional scene, the mediod comprising the steps of: 

(a) rendering said three dimensional scene so as to provide at 
least one psychological vision cue; 
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(b) generating image data corresponding to a set of two 
dimensional image slices through said three dimensional scene; 

(c) providing said image data corresponding to said set of two 
dimensional slices of said three dimensional scene to an image projector; and 

5 (d) selectively projecting each of said two dimensional slices 

from the image projector onto an optical device to generate a first volimietric 
three dimensional image, wherein said at least one psychological vision cue 
causes the apparent deptii of said first volumetric three dimensional image to 
exceed the physical depth of said optical device. 

10 33 . The method of claim 32 wherein said at least one psychological 

cue is provided by adding shading during rendering of said three dimensional 
scene. 

34. The method of claim 32 wherein said at least one psychological 
cue is provided by adding shadows during rendering of said three dimensional 

15 scene. 

35. The ifiethod of claim 32 wherein said at least one psychological 
cue is provided by occluding portions of said scene during rendering of said 
three dimensional scene. 

36. The method of claim 32 wherein said at least one psychological 
20 cue is provided by adding forced perspective during rendering of said tiiree 

dimensional scene. 

37. The method of claim 32 wherein said at least one psychological 
cue is provided during rendering by decreasing resolution in sxirface tejctures of 
more distant portions of said three dimensional scene. 
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38. The method of claim 32 wherein said at least one psychological 
cue is provided during rendering by increasing haziness of more distant 
portions of said three dimensional scene. 

39. The method of claim 32 whereui said at least one psychological 
5 cue is a provided during rendering by shifting the color of more distant 

portions of said three dimensional scene towards blue. 

40. The method of claim 32 v\rtierein said at least one psychological 
cue is provided during rendering by reducing the brightness of more distant 
portions of said three dimensional scene. 

10 41 . The me±od of claim 32 wherein said at least one psychological 

cue is provided during rendering by reducing the focus of more distant portions 
of said three dimensional scene. 

42. The method of Claim 32 wherein said optical device includes a 
plurality of optical elements. 

15 43. The method of Claim 42 v/herein said plurality of optical 

elements are arranged along a depth axis. 

44. The method of claim 32 further including a-floating image 
generator for projecting the first volumetric three dimensional image to 
generate a second volumetric three dimensional image viewable as floating in 

20 space at a location outside of the optical device. 

45. A method for generating a volimietric three dimensional image 
using a multi-surface optical device comprising a plurality of optical elements, 
said method including the steps of: 



173357.1 



wo 03/017246 



PCTAJS02/26718 



-70- 

(a) rendering a three dimensional scene having a depth axis; 

(b) generating data corresponding to a set of two dimensional 
image slices at varying depths dirough said three dimensional scene, each of 
said two dimensional image slices being defmed by a plurality of tridels each 

5 having a depth coordinate (z); 

(c) mapping the depth coordinate z of each of said tridels into 
a corresponding depth coordinate z' withm said multi-surface optical device; 

(d) mapping the depth coordinate z' of each of said tridels to 
at least one voxel having a depth coordinate k corresponding to one of said 

10 plurality of optical elements; 

(e) selectively projecting said voxels at said depth coordinate 
k onto a corresponding optical element to generate a first volumetric three 
dimensional image viewable in the multi-surface opdcal device. 

46. The me±od of daim 45 further including the step of projecting the 
15 first volumetric three dimensional image to form a second volumetric three 
dimensional image viewable in a space outside of said multi-surface optical 
device. 
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